Docs: Serving Layer

The Serving Layer is a Java HTTP server application, distributed as a runnable JAR file. This server implements a REST-ful API that exposes all methods of the recommender system. The server may be accessed over HTTP, or securely over HTTPS, and can be password-protected so that access requires credentials using HTTP DIGEST authentication.

The Serving Layer can operate in two modes: stand-alone or distributed.

Stand-alone Mode

In stand-alone mode, one Serving Layer instance operates as the complete system on one server. It performs all computation locally, and does not access a distributed Computation Layer. All input and intermediate data is stored locally in a directory on the local file system. Here, the Serving Layer is not clustered either, and only a single instance can be run at once.

It makes use of a local working directory to record input it receives, and to store the results of intermediate computations. Because its input files are simple text files with comma-delimited fields, it's fine to simply put new input into its working directory at any time. Each line contains an integer user ID and item ID (e.g. 35235,98). The file name should end in .csv (or .csv.zip or .csv.gz if compressed).

Choose a writable working directory on a local file system, and place any initial input in this directory. Specify it at startup with the --localInputDir flag. The Serving Layer instance will compute a model and save it as the file model.bin in the directory. It will be updated when the model is rebuilt. If the file is present at startup, it will be read to restore the server state, rather than re-reading CSV input in the directory and recomputing the model. Thus the file can be saved and restored as a way of preserving and recalling the server's state of learning.

To start the server, download the myrrix-serving-x.y.jar file and then invoke it as follows:

java -jar myrrix-serving-x.y.jar --localInputDir /path/to/working/dir --port 8080

See the Tuning Guide for more suggested JVM flags. The following additional command line options are available:

Flag Description Default
--localInputDir The local directory used for reading input, writing output, and storing intermediate model files in local mode. System temp dir
--port Port on which to listen for HTTP requests. Note that the server must be run as the root user to access port 80. 80
--securePort Port on which to listen for HTTPS requests. Likewise note that using port 443 requires running as root. 443
--contextPath URI base for endpoint URIs; defauls to none, or the root context. Not recommended, but if set too /foo, will cause the recommend method endpoint, for example, to be accessed at /foo/recommend instead of /recommend
--readOnly If set, disables methods and endpoints that add, remove or change data false
--keystoreFile File containing the SSL key to use for HTTPS. Setting this flag enables HTTPS connections, and so requires that option --keystorePassword be set. In distributed mode, if not set, will attempt to load a keystore file from the distributed file system, at sys/keystore.ks
--keystorePassword Password for keystoreFile. Setting this flag enables HTTPS connections.
--userName If specified, the user name required to authenticate to the server using HTTP DIGEST authentication. Requires password to be set.
--password Password for HTTP DIGEST authentication. Requires userName to be set.
--consoleOnlyPassword Only apply username and password to admin / console pages. Not set / false
--hostRequestLimit Max number of requests per minute from a host before it is temporarily blocked. This provides only a basic attempt to deny requests and is not guaranteed to block any DoS attack. Optional.
--rescorerProviderClass Optional. Name of an implementation of RescorerProvider to use to rescore recommendations and similarities, if any. This may also be specified as a comma-separated list of class names, in which case all will be applied, in the given order.

The class file or JAR containing it must be added to the server class path. The usual java command must be modified to specify an additional jar file:

  • Add -cp myrrix-serving-x.y.jar:your.jar to the command line
  • Replace -jar myrrix-serving-x.y.jar with net.myrrix.web.Runner

Or, in distributed mode, if not found in the classpath, it will be loaded from a JAR file found on the distributed file system at sys/rescorer.jar.

--clientThreadClass Optional. Name of an implementation of net.myrrix.online.ClientThread which is intended to be run in the Serving Layer in its own thread as an in-process "client" of external services. This may be used to poll/pull updates from some external service and push directly into the recommender, or perform any other service that a caller needs. The thread will be started with the web container and closed with the web container.

See notes above describing how RescorerProvider code is added to the classpath. The same applies for ClientThread. In distributed mode, it is loaded from a JAR file found on the distributed file system at sys/clientthread.jar.

The following system properties may also be set on the command-line using Java's -Dkey=value syntax. These must appear before -jar. These are advanced settings that are usually not necessary to change.

Key Description Default
model.iterations.max A hard limit of the number of iterations that will run in one build. 30
model.features Number of features to use when creating the matrix factorization 30
model.als.iterations.
convergenceThreshold		 Estimated strength values in the original matrix change a little after each iteration, and less over time. If average absolute change in estimates is below this threshold, iteration will stop. 0.001
model.als.lambda Controls the lambda overfitting parameter in the ALS algorithm. This should not normally be changed. 0.01
model.als.alpha Controls the alpha parameter in the ALS algorithm. This should not normally be changed. 40
model.noKnownItems If true, does not store in memory items that each user is already associated to. This saves memory, but means that the recommender does not remember which items the user is already associated to. These can't be automatically removed from consideration as recommendations. This is desirable behavior in some contexts. To use this, the considerKnownItems argument to recommend must be true. mostPopularItems will also not work with this flag enabled. Not recommended in general. false
model.local.
writesBetweenRebuild		 Sets the number of new data points written to the model that will trigger a model rebuild. From version 0.8 onwards. Only applies to stand-alone mode. 10000
model.distributed.
writesBetweenUpload		 Sets the number of new data points written to the model that will trigger an upload of local data to the distributed storage system. From version 0.8 onwards. Only applies to distributed mode. 50000
model.lsh.sampleRatio Enables locality-sensitive hashing to speed up the /recommend method, at the cost of accuracy. Set to a value in (0,1]; LSH is disabled unless set to a value less than 1. Recommended values are less than 0.1. This feature is experimental. 1.0
model.candidateFilter.customClass Specifies an implementation of net.myrrix.online.CandidateFilter that is available on the classpath. This class contains logic that enumerates which subset of all item IDs are considered for recommendation. It is used to speed up recommendation requests. This can't be used with model.lsh.sampleRatio and will override it. See RescorerProvider above for information on how to include an implementation of this class in the Serving Layer. This feature is experimental.

Distributed Mode

In distributed mode, one or more Serving Layer instances cooperate with a Computation Layer, sending it data and reading its computed recommender models. The Serving Layer operates nearly identically in this mode. Its API is the same, and all flags and parameters above are still applicable. A few more flags are available, described below.

The most significant difference is that each Serving Layer server is, potentially, just one of many. The recommender model and work load can be partitioned by user ID in distributed mode, and each Serving Layer can serve only one of many partitions. Multiple Serving Layers can serve one partition as replicas, for scalability.

The Serving Layer for distributed mode exists as a separate binary from the stand-alone mode binary available for download. The distributed mode Serving Layer binary is available for Amazon's S3 and Elastic MapReduce services (myrrix-serving-distributed-aws-x.y.jar), and for Apache Hadoop and HDFS or work-alikes (myrrix-serving-distributed-hadoop-x.y.jar).

These binaries accept a few additional command-line parameters:

Flag Description Default
--localInputDir The local directory used for staging input for upload in distributed mode. System temp dir
--instanceID Uniquely identifies the recommender from others that may be run by the same organization. Only applicable in distributed mode. Must be set with --bucket
--bucket Identifies the root directory of storage under which data is stored and computation takes place in distributed mode. Must be set with --instanceID
--partition Optional, but must be set with --allPartitions. The partition (0-based) that this is Serving Layer is serving.
--allPartitions Optional, but must be set with --partition. Describes all partitions, when partitioning across Serving Layers by user. Each partition may have multiple replicas. When running in distributed mode on Amazon EC2, may be specified as "auto", in which case it will attempt to discover partition members dynamically, searching for instances tagged with EC2 key "myrrix-partition" and whose value is a partition. (Port may be specified with EC2 tag myrrix-port if not the default of 80, and, instances may be uniquely associated to a bucket and instance with myrrix-bucket and myrrix-instanceID EC2 tags if needed.) Otherwise, replicas are specified as many Serving Layer "host:port" pairs, separated by commas, like foo:80;bar:8080;baz2:80. Finally, partitions are specified as multiple replicas separated by semicolon, like foo:80,foo2:8080;bar:8080;baz2:80,baz3:80
--licenseFile From version 0.9. Location of a license file named [subject].lic, where [subject] is the subject name authorized in the license. The license file should be valid at the time the app is run, and contain authorization to use the amount of parallelism (max simultaneous Hadoop workers) requested. Only required in distributed mode.

When using distributed mode with Amazon services, it is also necessary to supply an AWS access key and secret key as system properties, store.aws.accessKey and store.aws.secretKey. These may also be specified as the standard AWS environment variables, AWS_ACCESS_KEY and AWS_SECRET_KEY. Refer to Computation Layer documentation.

Console

The Serving Layer exposes a simple management console, which can be accessed with a web browser. For example, if the server is running on example.org, navigate to http://example.org. If HTTPS is enabled, navigate to https://example.org. If authentication is enabled, you will be asked to enter your specified user name and password.

The console has basic information about the machine on which it is running and its available resources, as well as the overall state of the recommender engine, such as number of users and items in the model. It reports the number of requests, and average request time (in µs) for different methods exposed by the API.

The console provides a rudimentary visualization of the data points -- user and items -- in feature space. Even the feature space is high-dimensional -- usually 30 or more dimensions -- and so cannot be displayed directly on a two-dimensional screen. The data are mapped onto a square 2D grid using self-organizing maps. This creates a visualization in which similar users or items, represented by dots on the grid, tend to cluster together, and regions of similar color contain similar items.

The map is generated randomly every time the visualization is loaded. It can visualize users or items. Mouse over the dots to see the IDs of points in each grid cell. Only a few representative IDs are shown for each cell.

Finally, the console also provides a simple means to execute requests against some of the API methods. For example, accessing /estimate/135/95 returns the estimated association from user 135 to item 95. This can be entered by filling in the template at the left. After clicking the arrow, the raw response from the server is displayed at the right. This can be used for simple debugging.

Utilities

A few utility classes simplify tasks which may be useful for testing: computing recommendations for all users, or similar items for all items. The Computation Layer can compute these as well, or, it is always possible to repeatedly query the Serving Layer for any or all recommendations or similarities.

Still, the program AllSimilarities exists to compute all item similarities, given an input directory like that for the Serving Layer. Likewise, AllRecommendations will compute recommendations for all users locally.

Javadoc

Browse Serving Layer Javadoc.