Colour theme
Request a demo

On-premise vs cloud AI: what changes for your data

Published Updated

On-premise AI, also called in-house or local AI, means that the AI model, and the application that uses it, run on hardware the company manages itself: a server in the office, a workstation, or the company’s own data centre. Its counterpart is cloud AI, where the model runs on a provider’s servers and the company reaches it over the internet. This guide puts the two side by side. Where are questions and documents processed, what does an on-premise setup gain, what does it cost, and which questions should you ask before you decide?

What does on-premise AI mean?

On-premise means that software or a service runs at the company’s own site, on hardware the company owns or rents and operates itself. For AI this means the work of producing an answer, called inference, is done on that hardware.

There is a subtlety: on-premise and private cloud are not the same thing. According to NIST’s definition of cloud computing, a private cloud is infrastructure provisioned for exclusive use by a single organisation and may exist on or off premises, while a public cloud exists on the premises of the cloud provider (NIST SP 800-145, September 2011). So the word “private” does not by itself tell you that the hardware sits in your own building; you have to ask where it stands.

Running on-premise has a second condition: the model must be available to download onto your hardware. Such models are called open-weight models: the trained model’s weight files are published and anyone can run them on their own hardware. “Open-weight” is not a synonym for “open source”. The Open Source Initiative’s Open Source AI Definition 1.0 (28 October 2024) asks for sufficiently detailed information about the training data and the code used to train and run the system, as well as the model parameters. Having downloadable weights does not mean these conditions are met, and the model’s licence still has to be read.

How does a question travel in the cloud and on-premise?

The difference shows best at the moment an employee asks the model a question and attaches a document.

Step Cloud AI On-premise
Where the question and the attached document go Leave the company network for the provider’s servers Go to the company’s own server or workstation
Where the model runs The provider’s data centre The company’s hardware
Who sets the rules The provider’s contract and settings The company, within the model’s licence
Maintenance and updates The provider’s job The company’s job
If something fails The provider responds; the company depends on the contract and the service The company responds with its own team or a support contract

This difference is neither good nor bad in itself; it moves responsibility. NIST’s guidelines on public cloud computing (SP 800-144, December 2011) describe how the terms of service can be changed unilaterally by the provider in some classes of service and can be negotiated in others, and they stress one point: the organisation that chooses an outside service remains accountable for its security and privacy. The guidelines are a general cloud framework, not specific to AI, but the question is the same: if your data is processed outside your network, monitoring and overseeing the provider remains your job.

Advantages of an on-premise setup

  • The processing location is known. While an answer is produced, the question, the document and the answer are processed on a machine in the company’s own network. Whether anything else goes out (web search, update checks, a call to a remote service) depends on the settings of the setup, and the company can see them.
  • The company sets the rules. Which model version runs, when it is updated and how the service is configured do not depend on a provider’s unilateral decision.
  • Less dependence. A provider’s decision on price, quota or retiring a model version does not directly affect your workflow.
  • It fits what exists. User accounts, network separation and backup, which the company already operates, can serve the AI as well.

Drawbacks and cost items

An on-premise setup hands back to the company the work a cloud provider would carry. All of it is a cost item:

  • Hardware. The memory and compute the model needs are bought or rented, and lead times are possible.
  • Operations. Security patches, software updates, backup, monitoring and incident response become the company’s work, so someone in the company or a support contract has to carry them.
  • Model choice. The models you can run are limited to those that fit your hardware’s memory.
  • Capacity. When the number of users grows, raising capacity often means adding hardware.
  • Responsibility. In a failure, data loss or security incident the company is its own provider.

Memory is often what decides the hardware. The room a model takes in memory is its parameter count times the number of bits each parameter is stored in; the same model stored in 4 bits instead of 16 takes a quarter of the room. The memory table in Hugging Face’s Accelerate documentation shows this in an example and adds that extra memory is needed while the model answers. So “which hardware?” cannot be answered without saying which model, and how many people will use it.

In between: managed cloud and hybrid setups

There are options between the two ends. In a managed setup a provider operates the infrastructure. If the infrastructure is reserved for your company alone, by NIST’s definition it is a private cloud outside the company; if it is shared with other customers, how the separation is achieved logically matters (the shared multi-tenant environment section of NIST SP 800-144). In a hybrid setup two or more infrastructures are used together (NIST’s definition of a hybrid cloud); for example, confidential documents stay in-house and the remaining work is done in another service.

In these options the company has no hardware of its own, but there is a party that runs the servers: the hardware and operations burden shrinks, and in return the place of processing is outside the company’s premises.

Which company fits which?

The following are general criteria; every company’s situation differs.

An on-premise setup generally stands out when:

  • Documents carry confidential information, trade secrets or personal data, and the company wants to know for itself where they are processed.
  • Use is steady and heavy and capacity can be predicted.
  • There is an IT team, or a support contract, to run servers and the network.

Cloud or a managed setup generally stands out when:

  • A quick start is wanted and an upfront investment is not.
  • Use is uneven or only an experiment.
  • Nobody is available to carry the operations.

A mix is also possible: sensitive work in-house, the rest elsewhere.

Eight questions to ask before you choose

Whichever way you go, ask the vendor or your own IT team for written answers.

  1. On which machine, run by whom, are my question and document processed? Ask about the country and any subprocessors too.
  2. What else goes out? Ask for the default state of features such as web search, update checks, usage metrics and calls to a remote model, and who can switch them off.
  3. Do the logs contain content? Ask whether error and usage logs carry the text of questions and answers; if content gets into logs, the logs are data that needs protecting as well.
  4. Who can see what? Ask whether permissions are granted per document and conversation, and whether an administrator can open someone else’s conversation.
  5. Which actions are recorded? Administrator actions and permission changes are not the same as records of who read a document; decide in advance which you need and ask for it in writing.
  6. How do backup and restore work? Ask where the backup is kept, whether it is encrypted and whether a restore has really been tried.
  7. How are model and software updates handled? Find out which model is used under which licence, who chose it, how often updates come and whether they can be rolled back.
  8. Whom do I call when something fails? Read the support hours, the response time and how responsibility is shared in the contract. According to NIST’s guidelines, with an outside service responsibility rests with the organisation in the end.

If personal data is involved, is an on-premise setup enough?

Not by itself. The Turkish Personal Data Protection Authority’s text on generative AI in the workplace (5 March 2026) says that Law No. 6698 (KVKK) is a general legal framework that applies wherever personal data is processed, regardless of technology, and that processing carried out through generative AI systems is assessed within it. The Authority’s guide of November 2025 says that if a company in Turkey uses generative AI systems through service providers based abroad and personal data is transferred abroad that way, the transfer must comply with Article 9 of the Law and the related regulation. An on-premise setup changes the questions that arise from data going to a provider; it does not remove the obligations of a company that processes personal data. For details, see the guide on KVKK and AI (in Turkish): KVKK ve yapay zekâ. This article is not legal advice.

How AURA works

AURA comes in three setups: Company Server (a workstation or server in the office), This Computer (a Mac with an Apple Silicon processor, macOS 26 or later and at least 16 GB of memory) and Voiser Cloud (Voiser runs the servers). In the Company Server and This Computer setups the models run on the company’s own hardware. Web search starts off on Company Server and This Computer and works if your organisation turns it on; on Voiser Cloud it comes on, and your organisation can turn it off. When it is on, only the search query and the addresses of the pages opened leave. On Company Server and This Computer, with web search off, questions, documents and answers are not sent out. In Voiser Cloud, for the organisation that wants it, Voiser hosts AURA on its own servers, Voiser runs the servers and the data stays on Voiser’s servers. That is a choice, not a flaw: there is no hardware or operations burden, and in return the place of processing is outside the company’s premises.

The models are open-weight, their licences are Apache-2.0 or MIT, and they are used as they are; AURA does not train or fine-tune any model on customer documents, and documents are indexed for search only. Administrator actions and permission changes are recorded, the record holds no content, and an administrator cannot open other people’s conversations or documents.

For more, see the on-premise AI assistant and AI for company documents, or request a demo.

Frequently asked questions

Does on-premise AI need an internet connection?

Because the model runs on the company's own hardware, producing an answer does not require sending a request out; but a connection is needed to fetch the model the first time and to update it. Features that connect outward, such as web search, are switched on or off separately.

What hardware does on-premise AI need?

It depends on the size of the model and on how many people use it at once. Memory is often the deciding factor: a model takes room in proportion to its parameter count and to the number of bits each parameter is stored in, and it needs extra memory while it answers. Try it with your own documents and your real user count before you buy.

What is an open-weight model?

A model whose trained weight files are published, so it can run on your own hardware. Models that also open their training data and code come closer to the Open Source Initiative's definition of open source AI; in both cases the licence still has to be read.

Is on-premise the same as a private cloud?

No. By NIST's definition a private cloud is infrastructure provisioned for a single organisation and may be inside or outside the company's premises; on-premise means the hardware sits on the company's own premises.

Sources

The outside facts in this guide rest on these sources. The date is the source’s publication date; for a live document with none, it is the date it was accessed.

  1. The NIST Definition of Cloud Computing (SP 800-145)Publisher: National Institute of Standards and Technology (NIST) · Date:
  2. Guidelines on Security and Privacy in Public Cloud Computing (SP 800-144)Publisher: National Institute of Standards and Technology (NIST) · Date:
  3. The Open Source AI Definition 1.0Publisher: Open Source Initiative · Date:
  4. Üretken Yapay Zekâ ve Kişisel Verilerin Korunması Rehberi (15 Soruda) [Generative AI and the Protection of Personal Data: a Guide in 15 Questions]Publisher: Kişisel Verileri Koruma Kurumu (Turkish Personal Data Protection Authority) · Date:
  5. İş Yerlerinde Üretken Yapay Zekâ Araçlarının Kullanımı [Use of Generative AI Tools in the Workplace] (announcement; the full text is in the document attached to it)Publisher: Kişisel Verileri Koruma Kurumu (Turkish Personal Data Protection Authority) · Date:
  6. Model memory estimator (Accelerate documentation; accessed 16 Nov 2026)Publisher: Hugging Face · Date:

See AURA with your own documents.

Request a demo