EnMasse User Guide

EnMasse is a server-based product for shared document formatting. It accepts documents over the network and formats them by distributing the formatting tasks among multiple computers. Regular users publish customized documents in high volumes and varied formats. Server access is via straightforward controls. The users may customize server operation for their work-flow.

Competing programs on the same computer fight for resources, and take more time to return a result.

More memory and newer processors help applications run faster, but there are often memory limits. Processors become faster, but their speed is limited; multiple-processor configurations are expensive, and performance does not rise in line with the price and the number of processors.

A solution is to distribute processing among multiple computers. If you have as many computers as there are programs, each program runs as fast as the computer allows it and uses as much memory as you can install. Using multiple computers, programs perform as well as the current level of technology permits, and can achieve a data throughput as high as you can afford; and the price for stock single-CPU PCs is low enough to have sufficient resources to meet your business needs.

If these are connected so that they work all the time, the load is evenly distributed, and even when nodes fail, or need upgrading and are removed, the system still processes all your requests, preserves your data, and the only change noticed is a slight decrease in performance. This is a well-known but not an easy task; and to really use the joint power of grid computing as it is known, you must resolve this issue.

XEP creates elegantly formatted print-ready views of documents. In many deployments, it produces hundreds of thousands of documents on demand. For example:

These are just a few examples. However fast and optimized the program's code, business needs a way to scale up performance so that it handles a growing load.

EnMasse solves this problem for you. For a user or an application programmer, it is a single access point. Through a shared folder, a web form or a network connection (locally on your network, or across the Internet), EnMasse accepts document processing requests and sends them to the XEP formatting engines running on several computers in the local network, then delivers formatted documents back to the user. It monitors the formatting engine performance, notices when they go down and are restored and readjusts the distribution of requests according to the workload. When a server fails in the midst of processing a request, EnMasse re-submits the request to a different server; thus the only impact is a slightly increased response time for that particular document. This provides the security of service needed in today's world.

EnMasse is both opaque and transparent. On one hand, it provides you with single point abstraction such that there is no need to worry about the number of engines running, or about their load; EnMasse dispatches requests to the most appropriate node. On the other hand, both the access point and the processing engines have standard, embedded servers. The system administrator can instantly check the status of the grid, identify problems and take appropriate action. An EnMasse access point takes little memory overhead and processing time, it can be deployed on a loaded intranet server, or even on a workstation and as long as the processing engines run on separate machines, it does not affect the speed or throughput of the grid.

For accounting purposes and performance tuning, EnMasse provides a logging facility. One can adjust the extent of the logging, or completely switch it off. The log files are easy to understand by humans and to process by programs.

EnMasse runs on a wide range of hardware and operating systems, easy to install and requires little maintenance. It has run for weeks on a mix of Linux, FreeBSD and Windows nodes wholly unattended, with multiple access points over the same grid when needed. It has proved to be the solution to many performance problems, providing a reliable service without high levels of support.

Actinia is basically a service of an Active Folder within the file system. Once an XSL-FO file appears in it (e.g., user drops a file using a normal file manager), Actinia notices it, picks it up, and sends it for formatting to one of servers in the grid. As soon as the formatting completes, Actinia then stores the formatted document in the output folder. The output folder can be the same as, or different from, the input one. This approach works when the user sends the document for processing, for example, when a different player needs the formatted document, or when the document leaves the system in another medium (for example, printed and delivered in a hard-copy form).

Toaster is a network-activated service. It monitors a network connection, accepts source (XSL-FO) styled documents, and sends back the formatted documents to the user via the same connection. Unlike the Actinia case, the client always receives the result of processing in an electronic form for local print generation. This is suitable when the user requesting document processing and is both the producer of XML sources and the consumer of their formatted output.

Fairy is a SOAP server which accepts source documents (XSL-FO) and sends formatted documents back via the same connection to the requestor (a client application). It can be easily tied with any application which supports SOAP, because writing SOAP clients is an easy task.

Fork is basically an integrator of an arbitrary number of services listed above (Actinia, Toaster, and Fairy). Its configuration file contains a list of config files, each of which configure corresponding EnMasse instances for each list entry. Fork then starts each listed instance. Additionally, it provides a Web interface for monitoring status of all started instances. Using Fork is an alternative to manually starting multiple EnMasse instances from console with their own config files passed as parameters.

EnMasse, in Actinia, Toaster and Fairy configurations, can apply XSL transformation to input documents. EnMasse nodes recognize xml-stylesheet processing instruction with type "text/xml" or "text/xsl" and apply the associated stylesheets to the source document. It allows to distribute both transformation and processing among the grid nodes, and fully utilize the power of the formatting framework.

For example, an installation dedicated to the formatting of DocBook documents may provide access to DocBook XSL stylesheets stored on a local server; the nodes will load and apply the stylesheets, and then format the generated XSL FO into PDF or PostScript.

EnMasse runs on any operating system that has TCP sockets, a Java Virtual Machine, and Python 2.7 or newer ( the current stable version at the time of writing is 2.7.14). Since you use XEP, you have Java. The installer of EnMasse is based on Python's Distutils module which comes with Python distribution. I recommend that you read Installing Python from the book Dive Into Python by Mark Pilgrim if you don't yet have Python on your computer.

Installing EnMasse in Toaster mode also requires a Web server able to run CGI scripts.

In order to install EnMasse, launch

python setup.py install

command from the command line. By default the package content will be installed in /usr/local/RenderX/EnMasse or C:\Program Files\RenderX\EnMasse directories, respectfully on Unix and Windows systems. You can change the default installation directory with --prefix option. For example:

python setup.py install --prefix=~/tmp

From now on the installed directory will be referred to as ${instDir}.

After installation, in order to use EnMasse you will need to configure it by creating working directories, log and configuration files. You can configure EnMasse either by hand or using a configuration script.

python setup.py configure

command creates all necessary working directories, log and configuration files. The complete list of working directories, as well as log and configuration files are described in the next sections. You can customize the configuration with --mode, --spool, --logs and --conf options. For example:

python setup.py configure --mode=toaster 
			--spool=~/tmp/EnMasse/spool
			--logs=~/tmp/EnMasse/logs
			--conf=/usr/local/RenderX/EnMasse/enmasse.conf

EnMasse installation has three types of installed content: programs and configuration files, working directories and program log.

Working directories.

EnMasse needs one folder as an internal working directory; additionally, Actinia requires three folders for user files: input , output and quarantine . It monitors the input folder for new work, places the formatted result into the output folder, and keeps a copy of all files not yet formatted in the quarantine folder so that if the system fails, all files are preserved and can be reprocessed. A temporary working directory, tmp is also used.

On Unix, these folders are in /var/spool/enmasse/; suggested folder names are inp/, out/, qua/, and tmp/. EnMasse must be able to write to the directories. All users^[1] who will be submitting files for formatting must be able to write to the input directory (and probably also to the output one so that they may delete the formatted files after retrieving them).

Program logs.

EnMasse writes detailed logs, to detect and resolve problems and tune performance. On Unix, /var/log/enmasse/ may be used to store them. The user running EnMasse must have write permissions for the logs' directory.

 python ${instDir}/lib/Python/enmasse.py etc/enmasse.conf 

 java com.renderx.xepx.cliser.Engine -DCONFIG=/path/to/xep.xml
            

To run EnMasse, you must configure it. A configuration file determines both how EnMasse interacts with the outside world and how it manages XEP engines and distributes the load. While for many parameters the default values are satisfactory, some values are required to be set explicitly to describe your local environment (the network and the computer).

EnMasse is packaged with a set of initial configuration files called actinia.conf.example, toaster.conf.example, fairy.conf.example, and fork.conf.example. Copying and editing example configs from ${instDir}/etc/ may be the fastest and the easiest way to start.

The configuration file is in the following XML format (in Relax NG). The complete specification of EnMasse configuration file follows:

config = actinia | toaster | fairy | fork
actinia = element actinia {
	actinia-folders & settings
}

toaster = element toaster {
	toaster-folders  & settings
}

fairy = element fairy {
	fairy-folders & settings
}

fork = element fork {
	fork-tines & element option { attribute name {"http-port"}, attribute value {string} }?
}

fork-tines = element tine {
		attribute conf {string}
	}+
	
settings = options & servers & cliser

actinia-folders = element folders {
	attribute input {string},
	attribute output {string},
	attribute quarantine {string},
	attribute temporary {string}
}

toaster-folders = element folders {
	attribute temporary {string}
}

fairy-folders = element folders {
	attribute temporary {string}
}

options = element option {
	attribute name {token},
	attribute value {string}
}*

servers = servers {
	element server {
		attribute host {token}?,
		attribute port {token}?
	}+
}

cliser = element cliser {
	attribute format {token}?,
	options
}

The EnMasse mode is set by the top-level element; it is toaster (network server), actinia (active folder), fairy (SOAP server) or fork(proxying mode). All modes except fork require folders and servers elements.

Attributes of folders are temporary, input, output, and quarantine which specify paths to the temporary folder (EnMasse kernel needs it), and, only for Actinia, to the input, output and quarantine folders. Actinia looks for XSL-FO sources in the inputfolder and writes formatted results to output. It keeps a copy of each XSL-FO source in the quarantine folder until processing is finished. This allows the user to re-submit documents after a system failure (for example, due to a power or hardware problem). Both input and output attribute values can point to the same location: Actinia picks files ending in '.fo' by default. You can set a different input filter. See the list of options below.

Here's a sample folders section:

     <folders
         temporary="'C:/EnMasse/tmp'"/>
            

servers defines servers available to the access point. For each server, host is the server's host name or IP address ('localhost' by default), and port is the XEP engine's port (the default value is 6570). You can list the same server multiple times if you want it to load it more heavily. XEP engines are multi-threaded and handle concurrent sessions efficiently.

Here's a sample servers section:

     <servers>
         <server host="'localhost'" port="6570"/>
     </servers>
            

CLISER, RenderX XEP Client-Server protocol, is the underlying protocol layer; element cliser sets the required document format (pdf is the default value, ps (for PostScript), or xep may be used), and can contain CLISER options (see the documentation on XEP for the list of option names). Use the same names as are available for XEP and prepend core options with 'FRM:' and generator options with 'GEN:' (optionally followed by the format's name and a colon). For example, the following fragment:

<cliser format="pdf">
  <option name="FRM:VALIDATE" value="'true'"/>
  <option name="GEN:pdf:COMPRESS" value="'false'"/>
</cliser>

sets output format to PDF, enables validation and switches off compression.

You can tune EnMasse' performance through a number of options. Default values are fine for most applications. By changing them you can build the exact configuration you want and fine-tune the load on the grid, the throughput, and the response time. Here is the list of all the available options, with their data types and default values in parentheses.

The protocol involves one request and one response. The client sends the request, in the form

RECEIVE data-size systemId
    			

followed by a zero byte ('\0' in C), and then by the data of data-size bytes in length itself. Toaster transforms the document into PDF or PostScript and returns it: it sends

RECEIVE data-size format
    			

followed by a zero byte and by the formatted document.

If EnMasse cannot format the document, it sends

ERROR message-size None
    			

followed by a zero byte and then by the error message. The message contains XEP's diagnostics and helps identify the problem.

To shutdown Toaster, send message STOP to the data port.

Fairy supports TLS (Transport Layer Security) that allows sending sensitive documents securely via unsecure networking transport. In this case, the clients are ensured they are connecting to an authentic server.

Fairy also supports two-way TLS, limiting access to authorized clients only. In this case, it stores public keys of all authorized clients and verifies the keys on each request.

EnMasse distribution contains a sample of Fairy client application. One may find it in etc\JavaClient\ within the EnMasse installation directory. The application shows how to use one-way and two-way TLS with Fairy.

Fairy also acts as a load balancer. Upon arrival, all new incoming formatting requests are internally queued. Once an Agent is available, it starts processing the first queued formatting request.

Queue length may be configured. If the limit is reached, e.g. the queue is full, and a new request arrives, it is rejected with Error 503 Service Temporary Unavailable and warning is logged. Since the queue keeps open connection sockets, the queue length is actually limited by the amount of free sockets in OS' TCP stack. Normally, this limit should not be reached, and the fact that it is being actually reached usually means that the configuration is insufficient for the purpose of effective load balancing.

Fairy will accept SOAP request without type specification for method's arguments. In this case format method's first argument will be interpreted as 'string' (and will be used without any conversion) and the second argument as 'base64Binary', but for greater compatibility it suggests the following types for arguments:

Here are examples of SOAP requests:

...
<format>
	<systemId>SYSTEMID</systemId>
	<xml>XML_DATABASE64ENCODED</xml>
</format>
...
				

...
<format>
	<systemId xsi:type="xsd:string">SystemId</systemId>
	<xml xsi:type="xsd:base64Binary">XML_DATA_BASE64_ENCODED</xml>
</format>
...