Routing and Load Balancing

If multiple Yarra servers have been installed in a network, it is possible to route certain reconstruction tasks to dedicated servers as well as to distribute the task load across multiple servers.

Server List File

To use multiple servers with the Yarra ORT client, it is necessary to create a server-list file named “YarraServerList.cfg” in the queue directory of the Yarra server that has been entered in the ORT configuration (preferably, the file should be placed in the queue directory of all Yarra servers and kept consistent). When opening the initial connection the the Yarra server, the ORT client will read the list of available Yarra servers and their properties from this file.

The file can be created and edited with a text editor and should have the following structure:

‘Server_ID’ is a unique ID that identifies the server, which must not contain spaces. A separate configuration section must be added to the file for each server, starting with the server ID enclosed by []. The entry “Type” defines the server type and can take a single value or a comma-separated list of server attributes. The ORT client will send reconstruction tasks to servers whose Type value matches the server type requested by the reconstruction mode. The entry “ConnectCmd” specifies the connect command that should be executed in order to map the queue directory of this particular server (similar to what has been entered in the ORT configuration). The command should map the queue directory under the same path as specified in the ORT configuration. Please note that the backslash character has to be written as “\\”. The entry “AcceptsUndefined” defines whether the server should receive reconstruction tasks without specified server type, or only tasks where a server type has been defined and the requested server type matches one of the attributes defined in “Type”. The entry “Disabled” can be used to temporarily disable servers from the task routing (e.g., if servers are down due to maintenance).
 

In the following example, two servers have been defined in the file YarraServerList.cfg, named 64core and gpu1. 64core has the attribute “multicore”, and thus, will accept all tasks that request a server of type “multicore”. In addition, it will also accept all tasks where no specific server type has been requested, as “AcceptsUndefined” has been set to true. The other server gpu1 will only accept tasks that explicitly request either the server type “gpu” or “research”.

It is highly recommended to write-lock the file after editing has been completed.
 

Assigning Reconstruction Modes to Servers

Reconstruction tasks can be assigned to Yarra servers in the configuration file queue/YarraModes.cfg, by adding the entry “RequiredServerType” to the section for the reconstruction mode. Alternatively, if the Yarra WebGUI is installed, the value can be instead added to the [ClientConfig] section of the .mode file (this is recommended).

Example: By adding the following [ClientConfig] section to the .mode file (and updating the file queue/YarraModes.cfg through the Yarra WebGUI), reconstruction tasks with this mode will be sent to a server that lists the attribute “gpu” in the “Type” value.

 

Load Balancing

If multiple servers of a type have been defined in the file queue/YarraServerList.cfg (or if multiple servers have been listed that accept tasks with undefined server type), the ORT client will cycle through the list of servers when selecting a server for a task. Using this simple “load-balancing” mechanism, the reconstruction tasks will be distributed across all available servers. Also, the mechanism provides inherent failover, as the client will contact servers in the list until a connection has been successfully established. Thus, if one of the servers goes down, the tasks will be seamlessly forwarded to the remaining servers.

At this time, the mechanism does not account for the number of already scheduled reconstruction tasks when selecting a server. Thus, it cannot be excluded that a new task is sent to a server where several tasks are already waiting in the queue. A more sophisticated load-balancing mechanism will be implemented in the future.