Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upFR Add tutorial on how to serve trained models from C++ #2054
Labels
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment


Originated as a forum post
Let's add a tutorial demonstrating how to serve trained models in C++ using torch.jit.save() and
torch::jit::load(). I believe there is some subtlety requiring atorch.jit.trace()to wrappoutine.traceandpoutine.replaylogic for SVI models and guides.Note that in some prediction tasks, only the
guideis needed and no Pyro logic is required. In this case, if the guide is annn.Module, it can be easilytorch.jit.save()ed andtorch::jit::load()ed without any need for Pyro.