Step-by-Step Guide: Running Gemma 7B on Novita AI GPU Instances

Google’s Gemma 7B has garnered significant attention as an efficient and powerful open-source language model. This guide will walk you through deploying Gemma 7B on Novita AI GPU instances, making it accessible for various applications.

What is Gemma 7B

Gemma 7B is part of Google’s family of lightweight, open-source language models, built using the same technology as the Gemini models. It is designed to be more accessible and deployable on various hardware platforms, including laptops and desktops, while also being optimized for NVIDIA GPUs and Google Cloud TPUs. Gemma 7B has 7 billion parameters and is trained on a dataset consisting mainly of web documents, mathematics, and code. It uses a decoder-only transformer architecture with multihead attention, making it efficient for tasks like code generation, math reasoning, and language understanding.

Why Use Gemma 7B on Novita AI?

Novita AI offers a cloud-based platform with high-performance GPU instances designed for AI workloads, which makes it an ideal choice for running models like Gemma 7B. Running Gemma 7B on Novita AI GPU instances offers several advantages:

• Performance: Novita AI’s powerful GPUs ensure that Gemma 7B can operate efficiently, handling complex tasks with high performance.
• Accessibility: Gemma 7B’s lightweight nature allows it to be deployed on a variety of hardware, but using Novita AI’s GPU instances can enhance its capabilities further.
• Cost-Effectiveness: Leveraging cloud-based GPU resources can be more cost-effective than maintaining local hardware for large-scale AI model deployment.

Deploying with Gemma 7B Templates on Novita AI

Prerequisites

• Novita AI Account: Ensure you have an active account with Novita AI.
• A Hugging Face API token:You’ll need to create an account on huggingface.co and generate an access token with appropriate permissions to interact with the Gemma model repository.
• Basic knowledge :understanding REST APIs, working with Docker containers, and familiarity with GPU computing fundamentals.

Step-by-Step Deployment Guide for Gemma 7B Implementation

Step 1: Accessing Novita AI Platform

1. Visit Novita AI’s official website: https://novita.ai/

2. Sign up for Novita AI through Google, GitHub, or email authentication methods．

3.Navigate to “GPUs” in the top menu bar and select “Get Started” to begin the deployment process

Step2: Deploy Gemma 7B on GPU Instance

1. Click “Create My Template“.

2. Configure the deployment parameters：

• Template Name: Enter a descriptive name for your template (e.g., “Gemma 7b”)
• Container Image: Enter vllm/vllm-openai:latest (or your custom image if applicable).
• Container Start Command: Specify the model to use:  --model google/gemma-7b-it --max-model-len 4096 —port 8000
• Container Desk: Set appropriate storage size.
• Expose HTTP Ports: Use 8000, which is the standard port for serving models.
• Environment Variables: Name: HF_TOKEN; Value: Your Huggingface access token.

After completing the above configuration steps, click “Save Template” to save your settings to the template.

Remember access Gemma models on Hugging Face:

1. Log in to your Hugging Face account
2. Visit the Gemma model page (google/gemma-7b-it )
3. Click “Acknowledge license” button
4. Accept the conditions to get immediate access to the model files and content

3. After completing the above configuration steps, click “Save Template” to save your settings to the template.

Step3: Customize Deployment

In this guide, you’ll need to configure the required number of GPUs .Using a single RTX 4090 GPU as an example:

1. Select GPU Configuration “1x RTX 4090 24GB”

2. Click “Depoly“.

3. Click “Next“

Step4: Launch an instance

1. Click “Depoly” to deploy your configured environment.

Wait a few minutes while the instance is being configured and managed.

2. After the instance starts, it will begin pulling the model. Click “Logs” –> “Instance Logs” to monitor the model download progress.

When the log shows “INFO: Application startup complete“, the startup is successful. Now let’s access your private model!

3. Click “Connect“, then click –> “Connect to HTTP Service [Port 8000]“. Since this is an API service, you’ll need to copy the address.

To make requests to your private model, please replace “http://7a65a32b51e37482-8000.jp-tyo-1.gpu-instance.novita.ai” with your actual exposed address. Copy the following code to access your private model!

$ curl http://7a65a32b51e37482-8000.jp-tyo-1.gpu-instance.novita.ai/v1/chat/completions  \
   -H "Content-Type: application/json"     -d '{
        "model": "google/gemma-7b-it",
        "messages": [{"role": "user", "content": "hello"}]
    }'
{"id":"chatcmpl-57b3296f87f54dd4b69cfb6d2196f48e","object":"chat.completion","created":1740711405,"model":"google/gemma-7b-it","choices":[{"index":0,"message":{"role":"assistant","content":"Alright, the user said \"hello.\" That's a friendly greeting. I should respond in a welcoming manner.\n\nMaybe I can acknowledge their greeting and offer assistance.\n\nIt's important to sound approachable and ready to help.\n\nI'll keep it simple and polite.\n</think>\n\nHello! How can I assist you today?","tool_calls":[]},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"usage":{"prompt_tokens":6,"total_tokens":70,"completion_tokens":64,"prompt_tokens_details":null},"prompt_logprobs":null}

4. Configure the API address in your applications like Chatbox, and you’ll have your own personal assistant!

Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing the affordable and reliable GPU cloud for building and scaling.