What is a local AI? A plain-English explanation

A local AI is an AI assistant that runs on a computer you own, instead of on a company's servers somewhere else. For an office, that usually means a computer in the office that your staff reach through a web browser.

How it differs from a cloud AI

With a cloud AI such as ChatGPT, your question travels over the internet to the provider's computers, which do the work and send the answer back. With a local AI, the AI model sits on a computer on your own network, and the question never has to leave it. Web search is the one optional exception, and it is off unless you switch it on. For more on this, see Private AI vs ChatGPT.

What it needs to run

• A computer with a powerful graphics card. The AI model lives in the graphics card's memory. More memory generally lets it run a larger model, handle longer documents and serve more people at once. Our builds are sized by this.
• An open-source AI model. These are models anyone can download and run. They are smaller than the biggest cloud models, but many are very capable at answering questions from documents, summarising and drafting.
• A web app on top, so staff can log in from a browser on the office network and ask questions. Nothing needs installing on their computers.

How it answers from your own files

Your documents are split into small pieces and indexed on the computer. When someone asks a question, it finds the pieces that match, and the model writes an answer from them. It does not retrain on your documents. It looks them up when you ask, and it can show which files it used. There is a longer explanation on how it works.

What a local AI is good at

• Answering questions from your own contracts, policies, reports and records.
• Summarising long documents and tidying or drafting emails.
• Keeping those documents and questions on a computer in your office.

What it is not

• Not the biggest model available. A local AI will not match the largest cloud models on every task.
• Not always right. Like any AI, it can get things wrong. Check important answers against the source document. It is not legal, financial or medical advice.
• Not available when the computer is off. It works on your office network, so working from home needs a secure remote connection.

Common questions

Does it need the internet? Your documents and questions are not sent out over the internet. The computer still needs to be on your office network, and the optional web search uses the internet.

Does it learn from our documents and share that with others? No. It looks up your files when asked. Nothing is shared with other offices.

Who is it for? Small businesses and offices that handle confidential documents and would rather keep them in the office, such as law firms and accountants.

Read more: How much does a private AI computer cost? and What to ask before you buy.