Novita AI LLM API

Empower Your AI Workflows with AnythingLLM and Novita AI

novita.ai

Nov 8, 2024 • 5 min read

Introduction

Excited to announce Novita AI’s support for AnythingLLM, a powerful all-in-one desktop platform that transforms how you interact with large language models. This integration brings Novita’s advanced language model capabilities directly to your desktop, offering a centralized hub for all your language processing needs.

One standout feature of AnythingLLM is its robust PDF processing function, which enables users to upload and analyze PDFs directly within the platform, as it allows for quick summarization, information extraction, and analysis of even the most complex PDF files.

By using Novita AI with AnythingLLM, you can transform PDFs into actionable insights instantly, saving time and enhancing productivity - all from a single, easy-to-use desktop interface.

What is AnythingLLM?

AnythingLLM is a full-stack application designed to give you the flexibility of using both commercial LLMs and popular open-source LLMs, along with vector database solutions. You can run this application locally or host it remotely, offering a secure, private environment to interact intelligently with your documents. By integrating these resources, AnythingLLM allows you to build a highly functional AI-powered chat interface that handles complex language tasks without compromising privacy or control.

Key Features

Thread-like Structure Workspace: Workspaces act as threads, organizing related documents in a contained space. Each Workspace isolates documents from others, preserving a unique context for each project or topic.
Flexibility for Personal and Businesses: Install AnythingLLM and its full suite of tools as a single application on your personal desktop. And Fine-grained permissioning and access control for organization is also built-in.
Security: AnythingLLM only talks to the services you connect to and can run fully on your machine without internet connectivity.

Why Should You Use AnythingLLM?

Multi-Model Compatibility: AnythingLLM supports both commercial LLMs like GPT-4o and popular open-source models like Llama 3.1 8b (Novita AI will help you on this), allowing users to select models based on budget, customization needs, and privacy concerns.
Local and Remote Hosting Options: It can run fully offline on a local machine or be hosted remotely, providing flexibility to maintain data security while still offering remote accessibility for team projects.
Workspace Management: “Workspaces” as mentioned above, acts like isolated threads, each organizing related documents separately, which keeps information compartmentalized and enables focused discussions on specific projects or topics without overlap.
Advanced Document Handling (including PDFs): The platform can import and analyze PDFs, allowing users to extract, summarize, and interact with document content directly, making it efficient for research-heavy workflows.

For instance, a featured and convenient private AI document chatbot can be easily deployed:

Launch a Private AI Document Chatbot in 10 minutes! | Open Source

How to Use Novita AI LLM API on AnythingLLM?

What Is Novita AI?

API, Serverless, and GPU Instance in one AI cloud. Novita AI provides cost-effective, seamlessly integrated solutions to accelerate your AI business.

Novita AI LLM API offers a highly cost-effective solution for developers, providing some of the lowest prices in the global industry. It deploys trendy models like llama-3-70b and llama-3.1-8b-instruct for high-speed production tasks, engaging discussions, and creative endeavors. The platform also features uncensored chat capabilities with popular models such as microsoft/wizardlm-2-8x22b and gryphe/mythomax-l2-13b.

Integration with AnythingLLM is streamlined through compatibility with the OpenAI API standard, and its API can auto-scale to handle changes in usage demand. Novita AI prioritizes reliability with fast network speeds and a commitment to data confidentiality by not storing prompt data for model training. Users also benefit from strong community support through an active Discord group, which fosters collaboration and assistance among users.

Step 1: Go to Novita.ai and Log In

You can log in on Novita AI with Google or GitHub, which will create a new account on your first login. Alternatively, you can register using your email address.

Step 2: Manage Novita AI LLM API Key

Novita AI secures API access through Bearer authentication by including an API Key in the request header, such as “Authorization: Bearer {API Key}”.

To manage your keys, navigate to “Key Management” in the settings. A default key is automatically generated upon your first login. You can create additional keys by clicking “+ Add New Key”.

Step 3: Download Desktop AnythingLLM

Go to AnythingLLM and download the latest version for your computer.

Step 4: Choose Novita AI as LLM Provider and Key in Novita API Key

Choose Novita AI from the LLM provider list in AnythingLLM settings. And then fill in your Novita AI LLM API key and set the model you want to use.

How to install AnythingLLM on Linux locally?

For Linux users, you can install AnythingLLM by running this command on terminal:

curl -fsSL https://s3.us-west-1.amazonaws.com/public.useanything.com/latest/installer.sh | sh

This will download the latest version of AnythingLLM’s AppImage, unpack it, and then supply a symlink to seamlessly run AnythingLLM. This script will unpack the app in $HOME/AnythingLLMDesktop.

You can start the app at any time by running ./AnythingLLMDesktop/start. This will boot the app with full logging.

Frequently Asked Question

Where are documents on Anything LLM saved?

When using AnythingLLM, all documents and data are securely stored on the device where AnythingLLM runs, whether it’s a local machine or AnythingLLM Cloud.

Why use AnythingLLM over LLM AI with a PDF plugin?

AnythingLLM grants complete control over access, permissions, and data sharing across any LLM and vector database, ensuring flexibility and security. It can run entirely offline, relying solely on local services, so privacy remains intact without third-party exposure.

How to use embedding models downloaded from hugging face?

Once you’ve downloaded a model, likely in GGUF format, you will need an LLM provider setup to run it effectively. LocalAI and LMStudio both offer environments where you can easily install and run models. Besides, we recommend you to try Novita AI, which also provides models for different scenarios.

How to use Dockerized Anything LLM?

You can refer to this page for how to storage locally and run AnythingLLM in Docker on Windows, MacOS, Linux or Docker Compose.

Conclusion

With Novita AI’s support for AnythingLLM, users gain a powerful, versatile tool that combines Novita’s advanced language models with AnythingLLM’s robust document management and secure workspace functionality. This integration empowers users to efficiently analyze PDFs, handle diverse data, and engage in complex, organized interactions — all within a privacy-focused environment that supports both local and cloud use. Whether for research, content creation, or data analysis, Novita AI and AnythingLLM together offer a comprehensive solution tailored to meet professional AI and document-handling needs.

Originally published at Novita AI

Novita AI is the All-in-one cloud platform that empowers your AI ambitions. Integrated APIs, serverless, GPU Instance — the cost-effective tools you need. Eliminate infrastructure, start free, and make your AI vision a reality.