Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Here is a sample shrine.conf file for a system running SHRINE 4.12.0 , for a node supporting researchers and distributing queries. 

...


Next, configure the initial network structures and queues for the hub.

Download the shrine network lifecycle tool from https://repo.open.catalyst.harvard.edu/nexus/content/groups/public/net/shrine/shrine-network-lifecycle-tool/4.12.0/shrine-network-lifecycle-tool-4.12.0-dist.zip into /opt/shrine:

...

SHRINE uses a message-oriented-middleware system to process queries and results asynchronously. Researchers can watch progress of their queries at different nodes, and see results as soon as any results are available. 

Starting in SHRINE 4.1 the The hub admin can select from SHRINE's own in-tomcat message system, AWS SQS, or Kafka. It is possible to switch from one MOM system to another in shrine 4.12, but the process is not gentle: Stop all nodes on the network. Configure all nodes for the new messaging system. Restart all nodes in the network.

...

For large networks that desire telecom-like reliability we use AWS SQS. AWS SQS stores messages that are in-flight or in-process, not the hub's tomcat. Messages stored in AWS SQS remain available if the hub stops. The hub processes those messages after it recovers. SHRINE 4.1 2 supports exactly one hub tomcat server; a shrine network can be no more reliable than the hub's tomcat. We hope to make the hub replicable in the future. Moving to AWS SQS now will make that transition more gentle.

To configure for AWS SQS see SHRINE 4.2.0 Chapter 8.2.1 - Configuring a Hub for AWS SQS .

Kafka

Shrine can also be configured to use Kafka. Using Kafka requires setting up, securing, and maintaining a cluster of Kafka servers - more admin expertise, attention, and expense than AWS SQS without the benefits of AWS SQS' reliability. 

To configure for Kafka see SHRINE 4.2.0 Chapter 8.2.2 - Configuring a Hub for Kafka .

Create the Network Record and Queues 

...