Feasibility and Usability Practice on Local Hosting Open Source Large Language Models (LLMs) Including Llama 3.2 Vision 90B in Multi-Functional Agentic Artificial Intelligence (AI) System to Drive Service for Design in the Latest Affordable Small Personal Computer (PC) System

Feasibility and Usability Practice on Local Hosting Open Source Large Language Models (LLMs) Including Llama 3.2 Vision 90B in Multi-Functional Agentic Artificial Intelligence (AI) System to Drive Service for Design in the Latest Affordable Small Personal Computer (PC) System

연구 분야: Software Development

학회: International Conference on Human-Computer Interaction

This paper aims to explore feasibility and usability practices from affordable small personal computer (PC) for local host of open source Large Language Models (LLMs) in multi-functional agentic artificial intelligence (AI) system on AI-driven services for design via a practice with experimental comparison view. Hence, a practical experiment method has been adopted for evaluating feasibility and usability practices for the local host of the systems. The results suggest that for the adopted three affordable small/mini PCs’ systems under price of 800 USD i.e., a Windows PC1 having dedicated graphic card with GPU and V-RAM, an Apple macOS PC Mac min m4 with 16GB RAM, and a Windows PC2 with inbuilt GPU and NPU with 96GB RAM, when LLMs models can be fully supported in the AI agent and agentic system, graphic generation performance is associated with the power of the graphic card with GPU and V-RAM, and the chat and vision performance is related to the power of the latest new hardware technologies for CPU, GPU, NPU and RAM. The Windows PC1 performs the fastest on graphic generation in Flux; and the Apple macOS PC Mac min m4 with 16GB RAM is the best to conducts chat and vision tasks via Msty. Interestingly, only the Windows PC2 can execute the biggest Llama3.2 vision 90b model for chat and vision tasks having the similar performance with running on the small Llama3.2-vision: latest 11b model. Moreover, all the three PCs can conduct multimodal understanding tasks of the latest Deepseek Janus Pro 7B model, whist in order to perform the tasks of multimodal text-to-image generation, the technical Torch program related issue needs to be addressed in the initial installation stage. Further, all the three PCs can local host the complex AI agentic systems such as Langflow, Bolt.diy, n8n, Flowise, and Dify with full functions working smoothly in the same conditions for the use of multi-functional production.