Interface#
Communication between model server and client is based on a simple HTTP protocol. It is closely modeled after the a mathematical function taking parameter vectors to model outputs, optionally providing derivatives like Jacobian action etc.
Inspired by the MIT Uncertainty Quantification library (MUQ), each model may have multiple input vectors and multiple output vectors. This allows for a cleaner separation between parameters of different purpors, but especially allows coupling multiple models in a graph structure (see MUQ’s model graphs), where the output of one model can serve as the input of another. Derivatives can then be traced through the entire graph of coupled models by applying the chain rule. The UM-Bridge protocol and integrations therefore always consider derivatives with respect to a specific input and a specific output.
Inputs and outputs to each HTTP endpoint are in JSON format as defined below.
Protocol#
Verifying correctness#
You can verify that your own model server fulfills the protocol definition by running the following test, adjusting the URL to where your model is running.
docker run -it --network=host -e model_host=http://localhost:4242 linusseelinger/testing-protocol-conformity-current
Required endpoints#
The model server offers the following endpoints:
Endpoint |
Purpose |
---|---|
/InputSizes |
Model input dimensions |
/OutputSizes |
Model output dimensions |
/Info |
Server protocol version and list of available models |
/ModelInfo |
Model specific information, in particular what optional endpoints are supported |
Optional endpoints#
The model server may offer the following optional endpoints, depending on the model’s capabilities. Whether an optional endpoint is supported or not is indicated in the output of the Info endpoint.
Endpoint |
Purpose |
---|---|
/Evaluate |
Model evaluation |
/Gradient |
Gradient of arbitrary objective of model output |
/ApplyJacobian |
Action of model Jacobian to given vector |
/ApplyHessian |
Action of model Hessian |
POST /InputSizes#
Input key |
Value type |
Purpose |
---|---|---|
name |
String |
Name of model to query |
config |
Any |
Optional and model-specific JSON structure containing additional model configuration parameters. |
Output key |
Value type |
Purpose |
---|---|---|
inputSizes |
Array of numbers |
Model input dimensions |
Input example:
{
"name": "forward",
"config": {}
}
Output example:
{
"inputSizes": [4]
}
POST /OutputSizes#
Input key |
Value type |
Purpose |
---|---|---|
name |
String |
Name of model to query |
config |
Any |
Optional and model-specific JSON structure containing additional model configuration parameters. |
Output key |
Value type |
Purpose |
---|---|---|
outputSizes |
Array of numbers |
Model output dimensions |
Input example:
{
"name": "forward",
"config": {}
}
Output example:
{
"outputSizes": [25,1]
}
GET /Info#
Output key |
Value type |
Purpose |
---|---|---|
protocolVersion |
String |
Protocol version |
models |
Array of strings |
Names of models available on this server |
POST /ModelInfo#
Input key |
Value type |
Purpose |
---|---|---|
name |
String |
Name of model to query |
Output key |
Value type |
Purpose |
---|---|---|
support : Evaluate |
Boolean |
Whether model supports Evaluate endpoint |
support : Gradient |
Boolean |
Whether model supports Gradient endpoint |
support : ApplyJacobian |
Boolean |
Whether model supports ApplyJacobian endpoint |
support : ApplyHessian |
Boolean |
Whether model supports ApplyHessian endpoint |
Input example:
{
"name": "forward"
}
Output example:
{
"support": {
"Evaluate": true,
"Gradient": false,
"ApplyJacobian": false,
"ApplyHessian": false
},
}
POST /Evaluate#
Input key |
Value type |
Purpose |
---|---|---|
name |
String |
Name of model to query |
input |
Array of array of numbers |
Parameter for which to evaluate model, dimension defined in /GetInputSizes |
config |
Any |
Optional and model-specific JSON structure containing additional model configuration parameters. |
Output key |
Value type |
Purpose |
---|---|---|
output |
Array of array of numbers |
Model evaluation for given input, dimension defined in /GetOutputSizes |
Input example:
{
"name": "forward",
"input": [[0, 0, 0, 0]],
"config": {}
}
Output example:
{
"output":[[0.10000000000000056,0.10000000000000052,0.1000000000000005,0.1000000000000005,0.10000000000000055,0.30000000000000165,0.3000000000000017,0.30000000000000165,0.3000000000000017,0.3000000000000017,0.5000000000000022,0.5000000000000023,0.5000000000000022,0.5000000000000023,0.5000000000000026,0.7000000000000018,0.7000000000000016,0.7000000000000021,0.7000000000000026,0.7000000000000028,0.9000000000000007,0.9000000000000008,0.900000000000001,0.9000000000000009,0.9000000000000012],[0.016300154320987727]]
}
POST /Gradient#
Input key |
Value type |
Purpose |
---|---|---|
name |
String |
Name of model to query |
inWrt |
Integer |
Index of model input to differentiate with respect to |
outWrt |
Integer |
Index of model output to consider in the objective |
sens |
Array |
Gradient of the objective with respect to model output |
input |
Array of array of numbers |
Parameter for which to evaluate model, dimension defined in |
config |
Any |
Optional and model-specific JSON structure containing additional model configuration parameters. |
Output key |
Value type |
Purpose |
---|---|---|
output |
Array of numbers |
Gradient of objective |
POST /ApplyJacobian#
Input key |
Value type |
Purpose |
---|---|---|
name |
String |
Name of model to query |
inWrt |
Integer |
Index of model input with respect to which the Jacobian should be taken |
outWrt |
Integer |
Index of model output which is to be differentiated |
vec |
Array |
Vector to apply the Jacobian matrix to, dimension defined in the |
input |
Array of array of numbers |
Parameter at which to evaluate the Jacobian, dimension defined in |
config |
Any |
Optional and model-specific JSON structure containing additional model configuration parameters. |
Output key |
Value type |
Purpose |
---|---|---|
output |
Array of numbers |
Jacobian of output |
POST /ApplyHessian#
Input key |
Value type |
Purpose |
---|---|---|
name |
String |
Name of model to query |
inWrt1 |
Integer |
Index of first input to differentiate with respect to |
inWrt2 |
Integer |
Index of second input to differentiate with respect to |
outWrt |
Integer |
Index of model output to consider in the objective |
vec |
Array |
Vector to apply the Hessian matrix to |
sens |
Array |
Gradient of the objective with respect to model output |
input |
Array of array of numbers |
Parameter at which to evaluate the Hessian, dimension defined in |
config |
Any |
Optional and model-specific JSON structure containing additional model configuration parameters. |
Output key |
Value type |
Purpose |
---|---|---|
output |
Array of numbers |
Hessian at |
Errors#
Each endpoint may return errors, indicated by error codes (i.e. 400 for user errors, 500 for model side errors) and a JSON structure giving more detailed information. The following error types exist:
Error type |
Description |
---|---|
InvalidInput |
Input does not match model’s dimensions or is otherwise invalid |
InvalidOutput |
Model delivered output not matching its own declared output dimensions |
ModelNotFound |
Model with given name not provided by server |
UnsupportedFeature |
Model does not support the requested feature (i.e. Evaluate, ApplyJacobian, etc.) |
JSON output then has the following shape, indicating error type and a specific message:
{
"error": {
"type": "InvalidInput",
"message": "Input parameter 1 has invalid length! Expected 64 but got 67."
}
}