Skip to content

docs(patterns): add vLLM Ray Inference Server sandbox pattern - #751

Open
BwL1289 wants to merge 1 commit into
validatedpatterns:mainfrom
eugo-inc:add-vllm-ray-inference-server
Open

BwL1289 wants to merge 1 commit into
validatedpatterns:mainfrom
eugo-inc:add-vllm-ray-inference-server

Conversation

@BwL1289

@BwL1289 BwL1289 commented Nov 8, 2026 •

Copy link
Copy Markdown

Summary

This pull request adds the vLLM Ray Inference Server pattern to the site in the sandbox tier.

The pattern deploys an OpenAI-compatible LLM server on one OpenShift cluster. One pod runs Ray Serve LLM with the vLLM engine on one NVIDIA GPU. The cluster builds the server image with a BuildConfig from hash-pinned PyPI wheels on the NVIDIA CUDA UBI 9 image. OpenShift GitOps deploys the server, the NVIDIA GPU Operator and Node Feature Discovery, Vault with the External Secrets Operator for an optional Hugging Face token, and a JupyterLab notebook that times the server.

We also emailed hybrid-cloud-patterns@googlegroups.com on 2026-11-08 to ask for the informal technical and
architecture reviews that the sandbox tier asks for. If that list is no longer read, or if it is easier, this pull
request can carry those reviews.

Files

File What it is
content/patterns/rhvp-vllm-ray-inference-server/_index.adoc Overview: background, architecture, deployed components, the AWS cost warning, and the support statement
content/patterns/rhvp-vllm-ray-inference-server/getting-started.adoc Prerequisites, installation, a test request, the notebook, and uninstallation, including deleting the GPU machine set
static/images/rhvp-vllm-ray-inference-server/architecture.svg Architecture diagram, rendered from the Mermaid source in the pattern README

Sandbox tier requirements

Requirement Status
Implementation requirements The pattern uses the clustergroup chart, the Validated Patterns Operator, and idempotent imperative jobs stored in Git. Its repository is public.
Deploys onto a freshly deployed OpenShift cluster without modification A fresh install passed: ./pattern.sh make install from main at 1a781d1, with default settings, on a new installer-provisioned AWS cluster (OpenShift 4.22.18, us-east-1, three m5.2xlarge nodes, no cluster changes). The command's own Argo CD health check passed, with all seven Argo CD applications Synced and Healthy. The pattern added the GPU node, the cluster built both images, the server answered the OpenAI API, and the notebook ran against it with no errors. An earlier install, on another new AWS cluster, found one bug (OpenShift's builder records an empty CMD as /bin/sh -c null), which 1a781d1 fixes (README, What this pattern was verified against).
README states the business problem README, "The problem it solves"
pattern-metadata.yaml pattern-metadata.yaml
Architecture drawing README, Architecture, and architecture.svg in this pull request
Informal technical review Requested by email to the mailing list on 2026-11-08, or here
Informal architecture review Requested by email to the mailing list on 2026-11-08, or here
Support policy README, Support, and the Support section of _index.adoc

Notes for reviewers

  • The name can change. The directory, aliases, image folder, and links use the current repository name. Two
    questions are open with the Validated Patterns team: whether the rhvp- prefix is acceptable under the Red Hat
    trademark guidelines, and whether the repository moves to validatedpatterns-sandbox. A comment at the top of the
    front matter says which values to change together.
  • No partners entry. The pattern uses NVIDIA, Ray, and vLLM software, but no partner organization takes part in
    it. The front matter explains this in a comment. Leaving the field out also keeps the partner trademark notice off
    the page.
  • The diagram is an SVG file. The Mermaid render hook on the site handles Markdown pages only, so the AsciiDoc
    page uses an image. The SVG uses plain SVG text, with no embedded HTML.
  • Commands and names in the getting-started page match the pattern README and the AGENTS.md file in the pattern
    repository.

Checks

  • hugo, run in the container image that make build uses, builds the site with these files and exits 0. The two
    pages report no warning other than the site-wide notice that the optional rouge gem is missing.
  • Every AsciiDoc attribute in both pages resolves. The overview page shows the warning block and the diagram.

Add an overview page and a getting-started page for the
rhvp-vllm-ray-inference-server sandbox pattern, plus its architecture
diagram.
@openshift-ci

openshift-ci Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Hi @BwL1289. Thanks for your PR.

I'm waiting for a validatedpatterns member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant