Models: pre-trained & custom

Out of the box, Backprop supports a variety of pre-trained models that can be explored on our model hub.

We dedicate time into researching and benchmarking the latest models available, and integrating them into the Backprop API.

That way, you'll always have access to a curated list of state-of-the-art models.

However, you might want to upload a model of your own. We support your models just like we do ours:

Each model is exposed as a private API that deploys with zero downtime, scales automatically, and requires no maintenance, so you can focus on building great applications.

Uploading your model

So far, the other tutorial has covered using our out-of-the-box models and tasks. This tutorial will cover how you can upload a model of your own, making it accessible from both the sandbox and API. Preparing a model for deployment is easy -- all you need is a .zip file containing a few key components.

Model Initialization

We'll begin with the model: in this example, it will be a simple mock model in PyTorch, with just one layer. The first thing to do is set up the class and its initialization function.

import torch
import torch.nn as nn

class MyModel(nn.Module):
	def __init__(self):
		# Can't use super() calls
		nn.Module.__init__(self)
		self.layer = nn.Linear(10, 2)

Adding Task Functionality

Now it's time to start implementing some mock "functionality" -- tasks to support, and how they'll be performed.

It's important that your model is callable. As such, our next steps will be to write the __call__ function, as well as the supporting functions that will be used in each task.

Before that, we should determine what tasks this model should support. For this example, let's say this model can do:

  • Text vectorisation
  • Text classification
  • Custom

The custom task is, perhaps unsurprisingly, completely implementation-defined. This serves as a way for you to deploy models that do exactly what you need, outwith the scope of our out-of-the-box tasks, while still giving you the same level of functionality with the API.

Let's implement the tasks now.

import torch
import torch.nn as nn
import random

class MyModel(nn.Module):
	def __init__(self):
		# Can't use super() calls
		nn.Module.__init__(self)
		self.layer = nn.Linear(10, 2)

def classify_text(self, text, labels):
	return random.choice(labels)

def forward(self, x):
	# Do something with input
	output = self.layer(torch.rand(10))
	# Random output for demonstartion
	return output

def __call__(self, params, task="text-vectorisation"):
	if task == "text-vectorisation":
		# Input according to Backprop's task specification
		text = params.get("text")
		# Output must be json serializable
		return self.forward(text).tolist()
	elif task == "text-classification":
		# Input according to Backprop's task specification
		text = params.get("text")
		labels = params.get("labels")

		return self.classify_text(text, labels)
	elif task == "custom":
		# Custom task supports any parameter
		my_input = params.get("some_parameter")

		return my_input
	else:
		raise ValueError("Unsupported task!")

We can break down the code above. The call function takes a params dictionary, as well as a task string. The supplied task is used to determine how the model will process the params and return output.

For the tasks Backprop already includes, the params input schema must match that of the corresponding API endpoint. Although you're using a custom model, accessing it from a non-custom task endpoint will still expect adherence to the specified format.

For text vectorisation, your params only needs to contain a "text" key, with the text to vectorise. This model will then call its{" "}

forward function (which, here, just gives some random results), and return the output.

For text classification, params consists of both "text" and "labels". The model calls its classify_text function, which selects a random label, and returns it.

For custom, the input schema and processing is completely up to the user. For this example, it expects a key "some_parameter", and returns whatever the value of "some_parameter" is. This is the same format you'll use to send custom requests to this model via the API.

Saving the Model

That's actually all we have to do to prepare the model class itself. Now it's time to save it to a .bin file using dill.

import dill

# Initialise and put on CPU
model = MyModel().to("cpu")

dill.settings["recurse"] = True

with open("model.bin", "wb") as f:
	dill.dump(model, f)

That's it -- the model is ready to go!

Adding meta files

To finish, we'll need to create a couple of meta files. The first is a classic Python requirements.txt file. This just lists all the dependencies needed to run your model. In this case, the file is minimal, since the model uses PyTorch, and nothing else. It looks like this:

requirements.txt

torch

Be aware that if you include unnecessary packages in this list, they'll add bloat to your upload and cause slowdown when using your model. Finally, we need to make a config.json file. This file tells Backprop which tasks your model can perform, and contains a description that'll be shown on your dashboard.

config.json

{
	"description": "This is the tutorial model!",
	"tasks": ["text-vectorisation", "text-classification", "custom"]
}

Now that we've got everything we need, we can compress the model.bin, requirements.txt, and config.json into a .zip archive.

Uploading the model

Now, head to the dashboard models page. Click "add a new model", give your model a name, and drag-and-drop the archive into the popup.

Hit upload, and sit back while Backprop processes the upload and deploys it.

Model zip upload

You'll have to wait for the model to be built and verified. This can take a few minutes, but the status will change from "building" to "deployed" once it's ready.

Pipeline upload in progressPipeline upload complete

And that's it! The model has been deployed successfully, and it's ready to be used, both in the sandbox and via the API.

For examples with other frameworks such as Tensorflow or SKLearn, check out our deployments repo on GitHub.

Deploying an SKLearn model is a very similar process to the one shown here, relying on dill for serialization.

Tensorflow is slightly different, with the model being in H5 format along with some extra functions that define how to load and call the model.