Setup and installation of 'GPU Supported DeepSeek & Llama powered All-in-One LLM Suite' on GCP

Note: We provide free demo access for the “GPU Supported DeepSeek & Llama powered All-in-One LLM Suite.” To request a free demo, please reach out to us at marketing@techlatest.net with the subject “Free Demo Access Request - [Your Company Name]”

This section describes how to provision and connect to ‘GPU Supported DeepSeek & Llama powered All-in-One LLM Suite’ VM solution on GCP.

1. Open GPU Supported DeepSeek & Llama powered All-in-One LLM Suite listing on GCP Marketplace.

2. Click Get Started.

It will ask you to enable the API’s if they are not enabled already for your account. Please click on enable as shown in the screenshot.

• It will take you to the agreement page. On this page, you can change the project from the project selector on top navigator bar as shown in the below screenshot.

• Accept the Terms and agreements by ticking the checkbox and clicking on the AGREE button.

• It will show you the successfully agreed popup page. Click on Deploy.

• On deployment page, give a name to your deployment.

• In Deployment Service Account section, click on Existing radio button and Choose a service account from the Select a Service Account dropdown.

• If you don't see any service account in dropdown, then change the radio button to New Account and create the new service account here.
• If after selecting New Account option, you get below permission error message then please reach out to your GCP admin to create service account by following Step by step guide to create GCP Service Account and then refresh this deployment page once the service account is created, it should be available in the dropdown.

You are missing resourcemanager.projects.setIamPolicy permission, which is needed to set the required roles on the created Service Account

• Select a zone where you want to launch the VM(such as us-east1-a)
• Optionally change the number of cores and amount of memory. ( This defaults to 1 NVIDIA T4, 2 vCPUs and 7.5 GB RAM)

NOTE: While deploying the instance on GPU, if you encounter the quota exhaust error or you are unable to deploy the instance on GPU VM then please refer to our Request Quota on GCP Cloud Platform

• Optionally change the boot disk type and size. (This defaults to ‘Standard Persistent Disk’ and 250 GB respectively)
• Optionally change the network name and subnetwork names. Be sure that whichever network you specify has ports 22 (for ssh), 3389 (for RDP) , 80 (for HTTP) and 443 (for HTTPS) exposed.
• Click Deploy when you are done.
• GPU Supported DeepSeek & Llama powered All-in-One LLM Suite will begin deploying.

3. A summary page displays when the compute engine is successfully deployed. Click on the Instance link to go to the instance page .

4. On the instance page, click on the “SSH” button, select “Open in browser window”.

5. This will open SSH window in a browser. Switch to ubuntu user and navigate to ubuntu home directory.

sudo su ubuntu

cd /home/ubuntu/

6. Run below command to set the password for “ubuntu” user

sudo passwd ubuntu

7. Now the password for ubuntu user is set, you can connect to the VM’s desktop environment from any local windows machine using RDP or linux machine using Remmina.

8. To connect using RDP via Windows machine, first note the external IP of the VM from VM details page as highlighted below

9. Then From your local windows machine, goto “start” menu, in the search box type and select “Remote desktop connection”

10. In the “Remote Desktop connection” wizard, paste the external ip and click connect

11. This will connect you to the VM’s desktop environment. Provide “ubuntu” as the userid and the password set in step 6 to authenticate. Click OK

12. Now you are connected to out of box GPU Supported DeepSeek & Llama powered All-in-One LLM Suite VM’s desktop environment via Windows machines.

13. To connect using RDP via Linux machine, first note the external IP of the VM from VM details page,then from your local Linux machine, goto menu, in the search box type and select “Remmina”.

    Note: If you don’t have Remmina installed on your Linux machine, first Install Remmina as per your linux distribution.

14. In the “Remmina Remote Desktop Client” wizard, select the RDP option from dropdown and paste the external ip and click enter.

15. This will connect you to the VM’s desktop environment. Provide “ubuntu” as the userid and the password set in step 6 to authenticate. Click OK

16. Now you are connected to out of box GPU Supported DeepSeek & Llama powered All-in-One LLM Suite VM’s desktop environment via Linux machine.

17. To access the Open WebUI Interface, copy the public IP address of the VM and paste it in the browser:

Browser will display a SSL certificate warning message. Accept the certificate warning and Continue.

18. The VM also comes with Certbot Nginx plugin preinstalled. So if you have valid domain name (DNS) and your instance IP is configured to access that DNS then you can generate free Letsencrypt SSL Certificates to access the Open WebUI securely over HTTPS using that DNS and can avoid Browser warnings. To do so, connect via terminal and run

sudo certbot --nginx

This command will prompt you for valid DNS Name, provide the DNS Name associated with this instance.

19. Once your Certbot certificates are ready , you can navigate to any browser and access the Open Web UI using DNS Name securely.

20. Click Get Started on very first page. This will take you to registration page. Provide the details here and create your first admin account.

21. Now you are logged in to Open WebUI Interface. You can choose different preinstalled models from the dropdown and ask your queries.

• To check the GPU utilization of the process, you can run below command in the SSH terminal. This will show GPU utilization every 5seconds.

 watch -n 5 nvidia-smi

• Or you can run “ollama ps” command from the terminal to check if GPU is being used by an Ollama model.

Please note that when switching models in OpenWebUI using the dropdown menu, the previously loaded model remains in memory for few minutes, which causes delays in loading the new model. . The solution to this issue is either refresh the OpenWebUI browser page before switching models or manually stop the previous model from the terminal using:

ollama stop modelname

To see the running models in the terminal run

 ollama ps

22. You can also run various ollama models from the VM’s terminal. To list the installed models run -

ollama list

23. To run the specific model use below command

Replace modelname with actual name of the model, e.g qwen2.5:7b

ollama run modelname

24. To pull any new ollama model run -

ollama pull modelname

Once your model is pulled successfully , you can start using it.

For more details, please visit Official Documentation page