NVIDIA Trition部署

下载地址

Server：

https://github.com/triton-inference-server/server

client

https://github.com/triton-inference-server/client

GPU方式部署server：

  docker pull nvcr.io/nvidia/tritonserver:21.05-py3

  git clone https://github.com/triton-inference-server/server.git

  cd server/docs/examples

  ./fetch_models.sh

  docker run --gpus=all --rm -p8010:8000 -p8011:8001 -p8012:8002 -v/mnt/zhangliang35/code/github/triton/triton-inference-server/server/docs/examples/model_repository:/models nvcr.io/nvidia/tritonserver:21.05-py3 tritonserver --model-repository=/models

GPU方式部署client

  docker pull nvcr.io/nvidia/tritonserver:21.05-py3-sdk

  docker run -it --rm --net=host nvcr.io/nvidia/tritonserver:21.05-py3-sdk

  /workspace/install/bin/image_client -m densenet_onnx -u localhost:8010 -c 3 -s INCEPTION /workspace/images/mug.jpg