AIsaAIsaStudio 中文 Open AIsa Studio

Tutorials news

Daily picks of AI news from official blogs, media, researchers and developer communities, with summaries and links to the originals.

Oct 31Sat · yesterday1 pick · 13 stories

r/AI_Agents· Tutorials

We ran AI agents on a real rooted Pixel instead of APIs, here's what broke first

The author ran AI agents from several projects on a rooted Pixel in Milan for a few weeks, exposing the phone as tools via MCP + REST so agents could send WhatsApp and SMS, manage groups, browse the web, and post on LinkedIn, because the official WhatsApp Business API was too slow and heavy for small Italian businesses. What broke first was WhatsApp moving the "invite via link" and "add member" buttons in one update, breaking group actions for a day until accessibility selectors were patched. Other issues included shared-inbox ownership (tagging inbound messages with an owner), webhook retries causing duplicate replies 7 seconds apart (fixed with an atomic lock keyed on message id), a daily cap on first contacts to avoid number restrictions, and a lease system for the single screen, with …

r/LocalLLaMA· Tutorials

Lawyer runs DeepSeek-V4.1-Flash on 2x DGX Spark plus a 3090 box, hitting 2,600+ tok/s prefill to 256K and publishing the full stack

The author runs DeepSeek-V4.1-Flash-1017 at 2.75 bpw EXL3 with vLLM tensor parallelism across two DGX Sparks, while an EPYC box with 3090s serves Engram tables over RDMA and runs the DSpark speculative drafter. Prefill reaches 2,618 tok/s at 256K, structured output hits 322 tok/s at 4 streams (138 single), and the ~1.93M-token KV cache plus NVMe prefix cache cuts first token after a 150K prompt restart from 58.6 s to 2.58 s. Code, launch scripts, the bench tool and raw JSON are all public at https://github.com/ProprietaryLegal/DeepSeek-V4.1-Flash-2xDGX-Spark-x86-offload (prefix-cache code pending); the author concedes slower plain prose decoding and weaker long-context decode than MiaAI's and bertholomus's builds.

r/AI_Agents· Tutorials

OpenAI's MCP review rules on tool annotations, and why a correct hint still isn't security

Thomas Sarlandie of OpenAI discussed at AGNTCon + MCPCon North America in San Jose (Oct 22-23) what ChatGPT's plugin directory reviews in production MCP servers; per InfoQ (Oct 26), OpenAI rejects submitted servers unless every tool carries annotations. Annotations include readOnlyHint (true only if the tool cannot change state), destructiveHint (true for irreversible results), and openWorldHint (true for tools reaching the public internet, including read-only web search). The review guide notes that justification text in a submission cannot override annotations; you must fix the annotation, deploy, rescan and verify. Per the MCP spec, clients must treat annotations as untrusted, and OpenAI's docs say annotations don't replace server-side authorization, input validation, or confirmation f…

r/LocalLLaMA· Tutorials

Qwen 3.8 27b vs Flash Next on BFCL and LiveCodeBench for a 32GB VRAM setup

The author benchmarked Qwen 3.8 27b against Qwen Flash Next on a 5090 with 64GB DDR. Qwen 3.8 27b scored 81.51% on BFCL and 86.15% pass@1 on LiveCodeBench, beating Flash Next's 78.19% and 79.22%, with a clear edge on long-context and harder problems; Flash Next only led in short single-answer categories. The author keeps the 27b setup for its 4 concurrent agents and a 570k nvfp4 KV cache pool.

AWS Machine Learning· Tutorials

A five-stage playbook for managing Amazon Bedrock costs end to end for FinOps and platform teams

AWS published a playbook for managing Amazon Bedrock costs through a five-stage loop: attribute, observe, enforce, optimize, and operate. Cost attribution uses IAM principal tags written to CUR 2.0, with per-user session tags restoring user-level attribution behind shared gateways. Budget enforcement spans three tiers, from AWS Budgets alerts and IAM deny policies to synchronous gateway blocking with HTTP 429. Optimization focuses on intelligent prompt routing and prompt caching, with cache multipliers for Claude Sonnet 4.6 and other models, scorecard thresholds, and sample CloudWatch Logs Insights queries.

r/LocalLLaMA· Tutorials

Single Intel B65 with XeStrata Flash-Next 4-bit hits 60 tok/s on Qwen3.8-Flash-Next

The author benchmarked a single Intel B65 (32GB) running Qwen3.8-Flash-Next with XeStrata Flash-Next 4-bit, offloading expert weights to host RAM, reaching a median 62.4 tok/s with 2.1s TTFT for a single user. Throughput drops to 48.7 tok/s aggregate at 2 concurrent users and 44.6 at 4. Compared against an RTX A6000 with llama.cpp MTP-2 (52.8 tok/s single-user, 75.2 at 2 users), the author argues the Intel setup is competitively priced and worth a second look for entry-level builds. Full data and interactive charts: https://b65.thekaiser.us/by-model/qwen38-flash-next/xestrata-20261030/

r/LocalLLaMA· Tutorials

SenseNova-U1.5-8B-MoT in llama.cpp on an M5 Ultra: Q4_K_M hits 125 tok/s and catches a $1 receipt mistake

llama.cpp merged support for SenseNova-U1.5-8B-MoT on Oct 21. The author converted the official weights and benchmarked it on a Mac Studio M5 Ultra: Q4_K_M reaches 125 tok/s (Q8_0 88, BF16 56). The model is understanding-only (text or image in, text out); image generation requires stable-diffusion.cpp. In a vision test via llama-mtmd-cli, Q4_K_M spotted in 16 seconds that a fake grocery receipt's subtotal was $1.00 too high and that the 7% tax was computed from the wrong number, and it read back a menu, a whiteboard and three Chinese/English signs without mistakes. The author notes mradermacher's quants lack mmproj (no vision), that conversion fails with KeyError: 'vision_config' unless line 80 of conversion/sensenova_u1.py is changed to vision_config = self.hparams.get("vision_config", s…

r/OpenAI· Tutorials

PSA: Disconnect custom domains before moving ChatGPT from Personal to Business, or your Sites may be permanently locked

A user reports that upgrading a ChatGPT account from Personal to Business left their published ChatGPT Sites live but inaccessible, unable to be modified or deleted, with their custom domains still attached to the now-inaccessible workspace and unusable in the new one. OpenAI Support, after 10 days of waiting, said they lack the internal documentation to resolve the issue and no fix path currently exists. The author advises disconnecting custom domains and removing published Sites before migrating.

r/LocalLLaMA· Tutorials

Benchmark: Same 32 Models on M2 Pro, M3 Ultra, M4 Pro and M5 Max

A Redditor ran the same MLX benchmark on four Macs with identical settings, measuring decode speed for several models and Qwen-Image 2.1 generation. MoE models run as fast or faster on the M5 Max than the M3 Ultra since few params are active, while the dense 27B favors the Ultra's bandwidth; image generation is ~7x faster on M5 Max vs M4 Pro.

r/AI_Agents· Tutorials

October's 6 hottest models tested on the same agent coding tasks: the most hyped one passed 1 of 5

The author ran 5 small coding tasks with hidden traps against six models released in October, using Atomic Agent 0.6.13 in cloud mode via OpenRouter, with a 10-minute cap per task. DeepSeek V4.1 Flash, Claude Haiku 5.5, Gemini 4 Argon, Kimi K3.1 Pro and GPT-6.1 Terra all passed 5/5; DeepSeek V4.1 Flash cost $0.02 total with a 14 s median time, versus $0.65 for GPT-6.1 Terra, roughly a 30x price gap. The most hyped model, Recursive Flint (free tier, vendor-claimed 74.2 on SWE-bench Verified), passed only 1/5 and took 8 minutes on one task; the author notes its free endpoint kept erroring and returning 503s, so a rerun through a second agent never got a fair test.

r/AI_Agents· Tutorials

Hackathon build: a buyer agent negotiating against 8 seller agents, and what went wrong

For a hackathon the author built a group-buying system where one negotiator agent bargains against 8 seller agents, each with a private floor price and stock, submitting sealed offers. Money is handled by a deterministic engine rather than the models, and 200 random auctions ran with zero rule violations. Using Nemotron 3 Ultra (thinking off), failures included repeatedly calling already-answered read tools, messaging all sellers at once, ending turns with text instead of tool calls, and a smaller model returning prices as text like "$39.95"; fixes were removing answered tools, a fixed reminder message, and closing the string option in the schema. Every call uses a strict JSON schema with code-level re-validation, plus per-negotiation and daily spending caps that trigger a labelled replay…

PickHugging Face Blog· Tutorials

NVIDIA employee builds a grumpy MonsterHouse with Nemotron Voice Agent for Halloween

An NVIDIA employee shares MonsterHouse, a Halloween garage-door character that listens to trick-or-treaters and responds as a grumpy pirate ship, running the NVIDIA Nemotron Voice Agent on a single workstation. The pipeline uses Nemotron 3 ASR for speech, Nemotron 3.5 Bolt E4B 4GB (4-bit) with a LoRA adapter trained on over 5,000 multi-turn conversations for dialogue, Magpie TTS Zeroshot for seven-mood voice, and Audio2Face-3D plus Unreal Engine 5 for mouth animation, with Nemotron 3 Nano Omni, YOLO and ByteTrack for costume recognition and visitor tracking. Median end-of-speech-to-audio latency is 1.70 seconds (0.72 seconds cached), split across an RTX PRO 6000 Blackwell Max-Q and an RTX PRO 4000 Blackwell. The code is not packaged for release, but the post lays out a full path to rebuil…

The author breaks down the full stack from speech recognition and a local small model dialogue to lip-sync animation, with latency numbers that transfer to similar real-time character projects.

r/AI_Agents· Tutorials

How to read traces and run evals for agent debugging 101

The author teamed up with Antaripa Saha, who specializes in evals, search, and retrieval, to publish an in-depth tutorial covering how agent failures happen, how to find them in traces, and how to turn those failures into evals to make agents reliable in production. The author works on an agent reliability platform, posted the tutorial as an educational resource in the comments, and is open to questions.

Oct 30Fri11 stories

r/AI_Agents· Tutorials

How to build a harness that actually improves your agent, based on 57 papers

Summarizing 57 papers and personal experience, u/Marmelab lays out six harness practices: mine past sessions for rules, delete rules you can't prove help, keep instruction files short (OpenAI cut its main AGENTS.md to around 100 lines), use fewer tools (Vercel removed 80% of tools and success went from 80% to 100% with latency dropping from 724 to 141 seconds), back rules with scripts (research found only 4-16% of written security rules had real effect), and regularly prune stale rules. University of Toronto research shows the same model can see up to 6x performance gaps depending on harness design.

r/OpenAI· Tutorials

ChatGPT scheduled tasks consume your usage limit even when they post nothing

Citing OpenAI's tasks page updated this week, the author notes that completed scheduled tasks in ChatGPT consume your plan's usage allowance or workspace credits even when they post no update, since they still perform background model processing; OpenAI advises pausing unneeded tasks or running them less often. The page also states that data exports exclude scheduled tasks and their snapshots, deleting or archiving a chat can pause its tasks, share links must be deleted separately, locked chats can't hold tasks, project tasks can't access project files, and inactive tasks may pause on their own. Limits are 3 active tasks on Free and Go, 5 on Plus, 10 on Business and Edu, and 15 on Pro and Enterprise; Free tasks run at most once a day, while hourly runs and exact times require a paid plan.…

r/AI_Agents· Tutorials

100 live website tasks: a hands-on comparison of OpenClaw, Hermes, Claude in Chrome, and the 5dive browser

The author ran five browser setups on 100 live tasks from Online-Mind2Web, all using Claude Haiku 5.5 as the primary model: OpenClaw passed 35/100 (median 141s, $4.80), 5dive (claude code) 32/100, Claude in Chrome 29/100 but was fastest (median 67s), 5dive (pi) 29/100 and cheapest ($0.65), and Hermes 18/100; top differences were not statistically significant in paired tests. In 4 controlled routine tasks all setups went 12/12, with 5dive (pi) cheapest per 1,000 tasks ($0.86). The author also open-sourced a 5dive browser plugin supporting persistent logins, human takeover for logins and captchas, and approval before sensitive actions like payments, installable on self-hosted Debian/Ubuntu with two sudo commands; limitations included one model, one attempt, one datacenter IP.

r/AI_Agents· Tutorials

Team turns on CodeRabbit slop detection, 9 of 70 PRs flagged and sparks debate

The author's team enabled CodeRabbit's slop detection two weeks ago, which adds a walkthrough note and a "slop" label to PRs that look like low-effort AI output, such as 600 lines for a 20-line ticket, duplicate helpers, and tests that only check mocks. It flagged 9 of 70 PRs so far, 8 of which the author felt deserved it. The 9th was mostly hand-written by a junior developer who saw the label before anyone spoke to him, so the team changed it to show the label only after a human review. Half the team now wants it on every repo; the other half sees public "slop" labels as shaming with extra steps.

r/AI_Agents· Tutorials

How one team moved ~60 of their agents off a shared API master key to per-agent keys

The author's team migrated about 60 of their own agents off a single shared master API key to one key per agent. They built per-agent allow-lists naming exact routes with no wildcards (six of the first 25 were duplicates and merged), gave most agents read-only access with 403s for anything outside the list, and required a short-lived run (60 seconds, one call) per request so a runaway loop can't keep hitting the API. The master key was retired last, only after scanning every script and scheduled job found no references and each agent's own key had carried real traffic.

r/LocalLLaMA· Tutorials

10 coding agents tested on one Qwen 3.8 27B: pass rates similar, token use and cache reuse wildly different

The author ran Qwen 3.8 27B Q4 on llama.cpp (thinking off) on a MacBook Pro M5 Max (64 GB) and benchmarked 10 coding agents in sandboxes on 5 small coding tasks with built-in traps. Pass rates were similar across most agents, but token consumption varied hugely: Pi used about 9.2k tokens per task, the leanest, while Qwen Code used about 107.9k. Atomic Agent was the only one using llama.cpp's native endpoint with grammar-constrained tool calls, reusing 86% of cache, staying fully offline, and passing 5/5. Codex CLI, Cline, goose, and OpenCode attempted to reach the internet on every run even with telemetry off, blocked by the sandbox; DeepSeek Harness, still in alpha, passed 5/5, stayed offline, and had 83% cache reuse.

AWS Machine Learning· Tutorials

Epsilon builds intent-driven UI testing with Amazon Nova Act and Amazon Bedrock AgentCore Browser

Epsilon's Agentic Platform team delivers intent-driven UI testing as a multi-tenant platform service using Amazon Nova Act and Amazon Bedrock AgentCore Browser Tool, while keeping its existing Playwright suite as the regression backbone. Test steps are written in natural language, resilient to cosmetic UI changes and reviewable by non-engineers. The post covers the platform API, Celery-based task runner, and CDP WebSocket integration code, plus an 18-step example workflow with conditional logic. Per Epsilon's internal benchmarks, authoring the 18-step workflow took 5–10 minutes with Nova Act versus 30–90 minutes for an equivalent Playwright script (80–150 lines), a 4–6x effort reduction; replayed saved trajectories achieved a 100% pass rate. The team also shares practices on runtime isola…

r/AI_Agents· Tutorials

How to make a Claude Code rule enforced instead of just requested

The author points out that Anthropic's docs describe CLAUDE.md as context, not enforced configuration. Real enforcement should use permissions.deny rules or PreToolUse hooks that run before tool calls and can block them by exiting with code 2. The author also suggests a test: in a disposable repo, protect a dummy file and ask Claude to edit it to verify execution is actually blocked, noting that a check on one editing tool shouldn't be assumed to cover other tools.

AWS Machine Learning· Tutorials

How MoEngage turned over 400 APIs into agent tools with AgentCore Gateway

MoEngage used Amazon Bedrock AgentCore Gateway to turn roughly 400 internal REST APIs into permission-scoped agent tools serving 10 internal agents, saving an estimated six months of platform engineering effort and cutting new API onboarding from days to hours. Cedar policies filter tools/list server-side per principal, keeping the 20,000-plus-token tool catalog out of context windows; identity travels as X-MOE-* headers rather than tool parameters, with connectivity through a VPC interface endpoint and managed egress over Amazon VPC Lattice, no public endpoints anywhere. Product teams only add an x-moe-mcp annotation block to their OpenAPI specs, with CI handling target registration and policy application, so no team writes MCP server code or provisions networking.

r/AI_Agents· Tutorials

If you're giving an AI agent a wallet, put these guardrails outside the model

Drawing on crypto backend and agent-building experience, the author outlines guardrails for giving AI agents wallets: isolate private keys in a separate signing service, enforce limits and recipient allowlists in code rather than prompts, simulate transactions before sending, log intent and tx hash, and add a kill switch anyone on the team can flip. The post notes x402 has surpassed 100 million transactions and warns that web pages, emails, or API responses can carry prompt injection telling the agent to send funds, which is why limits cannot live in the prompt.

Oct 29Thu4 stories

r/AI_Agents· Tutorials

7 LLM eval tools compared: which one for what

The author compares 7 LLM eval tools in two groups: code libraries and trace-storing platforms. promptfoo handles YAML-based prompt regression tests and red-teaming, DeepEval fits pytest and CI, Ragas scores retrieval quality, Langfuse and Arize Phoenix offer open-source tracing, LangSmith pairs with LangChain/LangGraph, and Braintrust is a commercial platform with a free tier. The author advises hand-labeling a few dozen examples from real traffic before trusting any automated metric.

r/AI_Agents· Tutorials

How to check how much context your MCP tools use before the first message

The article explains how to check how much context MCP tool definitions consume before a conversation starts: some harnesses register every discovered MCP tool up front and include their descriptions and input schemas in each request. The author cites a report from a DeepSeek harness repo discussion where unmounting one large MCP server dropped resident context from about 200K to about 10K tokens…

r/AI_Agents· Tutorials

How to stop a browser agent from acting on a page that already changed: putting the check inside the engine

The author, building a browser where a human and an agent share the same tabs, says the biggest design challenge was staleness: by the time an agent's click on node 42 lands, a modal may have opened, the human taken over, or the page navigated. Their solution: every agent call carries a control epoch (who is driving the tab) and the page revision the plan was computed from; if the human took over or the document moved, the call fails with a stale target error and the agent must re-read, never acting on whatever sits at that position now. Opening, picking and closing tabs is human-only, so the agent prepares and the person commits. The cost is more round trips and treating stale as a normal outcome; the benefit is avoiding the silent wrong click that goes unnoticed until a form is submitte…

r/OpenAI· Tutorials

Export your personal ChatGPT before your company merges it into a workspace: no export button and no undo afterward

The author summarizes OpenAI's migration help page on merging a personal ChatGPT into a Business or Enterprise workspace: the merge is irreversible and self-service export is unavailable afterward; on Enterprise with a verified domain, organizations can require it with a 7-day deadline. Some content, such as older conversations, projects and GPTs owned by others, project-scoped memories, custom plugins and Sites, does not migrate. Since official export can take up to 7 days and the download link expires after 24 hours, the author recommends requesting it the same day the invite arrives and verifying it; the author's browser extension AI Toolbox exports chats directly from the page, with single-chat text export free and bulk ZIP plus Markdown, JSON and PDF formats as Premium features.

Oct 28Wed3 picks · 14 stories

r/AI_Agents· Tutorials

How to check whether your AI agent's security guard actually blocks what it flags

A security dashboard showing "prompt injection detected" doesn't mean the agent was stopped from using that text. Using StackOne's open-source Defender as an example, its README notes the blockHighRisk option defaults to false, so detection can flag risky content without producing a "not allowed" verdict. The author suggests checking four things: whether a high-risk finding actually produces a block verdict, whether the application enforces or just logs the verdict, what happens if an optional LLM reviewer refuses or returns malformed output, and whether you're trusting cleaned-up text instead of the verdict. The practical test is the full path: submit a known malicious tool result, inspect the verdict, and confirm the application stops it from reaching the agent.

r/LocalLLaMA· Tutorials

Tuning a single Tesla P40 for Ornith 1.5 35B-A3B: full 262k context at ~60 t/s short, ~30 t/s at 100k depth

The author tuned Ornith 1.5 35B-A3B (qwen3.5 MoE arch) on a single Tesla P40 24GB (PCIe x1) with a custom llama.cpp fork, holding Q4 weights and q8_0 KV rather than shrinking quants, at full 262k context. Working changes included disabling ECC (+1.5GB VRAM, ~9% decode), int8 dp4a for the flash-attention KQ kernel (prefill +26% at 32k, +41% at 100k), fixing the untrained MTP head (draft acceptance ~45% to 60-85%), and limiting MTP to the last 8k of the prompt (121k prompt: 327 to 373 t/s prefill). Without MTP, decode drops from 60.8 t/s at depth 0 to 29.1 t/s at 100k; with MTP in a real server, 65-74 t/s on short prompts and a 101k-token prompt reads at ~415 t/s and generates at 35-39 t/s. Patches are available on request.

Pickr/LocalLLaMA· Tutorials· 2 sources

DeepSeek AI Unlocks Consumer GPU Performance, Achieving Over 80% FP16 Speedup

A user successfully utilized DeepSeek AI to unlock the underlying Tensor Core functions of a consumer RTX 3080 GPU, bringing its performance close to the RTX A5000 level. The card saw its FP16/FP32 accumulation performance jump from 60.54 TFLOPS to 109.39 TFLOPS, realizing approximately an 80.7% performance leap. This process, which avoids hardware replacement, relies on the assistance of DeepSeek prompts.

Leveraging AI models to unlock consumer GPU performance achieved an over 80% boost in performance, showcasing its potential in hardware capability mining.

r/LocalLLaMA· Tutorials

Local AI app tuning: sustained inference cut from 100W to 25W on GPU

The author built a local-first discipline tracker around a 2B ternary model, where the camera captures photos that the local model evaluates to produce descriptions and activity reports. To fix the GPU constantly pulling 100W with loud fans during sustained inference, he lowered the power limit in the AMD Adrenaline panel and set fans to a constant low speed, cutting total draw to 25W while per-inference time went from 1.9s to 2.8s on average. He questions whether chipmakers ever tuned sustained Local AI workloads and asks what APIs exist to signal a sustained inference load to the platform.

r/LocalLLaMA· Tutorials

Testing 48 local models as agents on an M1 Max 64GB: qwen3.8-27b Q4_K_M comes out on top

The author built an unattended harness and tested 48 local models on an M1 Max 64GB using 16 multi-step tool tasks plus 10 rebuilt from real assistant failures (3 runs each), with speed measured for 54 models. qwen3.8-27b Q4_K_M was best overall (99% quality, 100% agent, 16.9 tok/s), ornith-1.5-35b-a3b is the fast pick (52.4 tok/s), and lfm2.5-24b-a2b passed all coding runs. 46 of 48 models made at least one serious slip on the real-world tier, and 5 models fell at least once for a prompt injection hidden in an email; on GGUF models a cached follow-up turn started 152x faster than cold start. Full tables, the harness, and traces are open-sourced at https://github.com/DesignwithHugo/LocalLlmBench.

r/LocalLLaMA· Tutorials

Built a Local GPU Benchmark Server to Test Real Performance of 4B to 35B Models

The author set up a local GPU benchmark server using two RTX 5060 Ti cards with 32 GB of VRAM running vLLM in Docker, and tested 12 local LLM models ranging from 4B to 35B. Tests cover 7 benchmarks including IFEval and IFBench, tracking wall clock duration, tokens per second, and calculating a time efficiency score. All results are published on the personal website at https://llmbenchmarks.io/local-gpu-benchmarks/.

r/AI_Agents· Tutorials

Same build task tested across 4 setups: single-agent vs multi-agent comparison

After a popular Reddit thread on single-agent vs multi-agent setups, the author gave the same Python expense tracker task to four setups from the comments: Claude Code (Opus 5.5) single agent, Codex CLI (GPT 5.5) single agent, a Paseo agent team (Opus 5.5 lead), and Atomic Agent Fusion mode (cloud Opus 5.5 planning, local Qwen 3.8 27B building). Each got the same one-line prompt and spec, graded by 38 hidden tests shown only after each run. Results: the three agent setups all passed 38/38 (cost ~$1.13–$1.32, around 6 minutes each), while Atomic Agent scored 36/38, 38/38 after a one-line fix, at $0.40 and 3m11s, fastest and cheapest. The Paseo lead judged the job small and built everything itself without spawning workers. The author's take: orchestration's value isn't making small jobs fas…

r/AI_Agents· Tutorials

I let an agent drive a real browser to record my product demo: clicking was easy, waiting was hard

To record a product demo for a YC application, the author used Playwright to drive a real Chrome window with OS-level screen recording and a fake cursor. Lessons include generating voiceover first, using a skip() to cut out non-deterministic waits (2-minute builds, 15-90 second ChatGPT turns), detecting completion by the Stop button being gone for 8 straight seconds, and replaying expensive agent steps instead of rerunning them. Pitfalls noted include ChatGPT's unsent drafts, nested iframes, and single-page apps overwriting document.title.

r/LocalLLaMA· Tutorials

Mojo and MAX enable inference on phones

Following the release of Mojo and MAX, local inference development is progressing smoothly. One user achieved on-phone inference results within 4 days, achieving performance comparable to or better than llama.cpp Vulcan.

PickAWS Machine Learning· Tutorials

How Property Finder Selected Llama 4 Maverick 17B Instruct on Amazon Bedrock Using Evaluation Bias Fixes and Prompt Optimization

Property Finder discovered that 96.8% of its property image quality assessment labels originated from predictions made by a single existing model, causing severe evaluation bias. By constructing unbiased ground truth and applying an evolutionary prompt optimization loop, they selected Llama 4 Maverick 17B Instruct on Amazon Bedrock. The final result saw the model's Micro F1 accuracy increase from 0.55 to 0.80, achieving an 86% cost reduction.

This case demonstrates how to identify model evaluation bias and use evolutionary prompt optimization to achieve model selection and cost reduction.

AWS Machine Learning· Tutorials

Exowatt beats frontier models on accuracy with SageMaker serverless model customization

AWS and Exowatt published a case study showing how Exowatt used Amazon SageMaker AI serverless model customization to fine-tune Qwen3-1.7B and Qwen3-14B for edge alarm triage and cloud energy-market analysis. The fine-tuned Qwen3-1.7B reached 0.911 macro-F1 on 320 test cases, beating Claude Sonnet 4.6's 0.698, while Qwen3-14B reached 0.975 versus Claude Opus 4.7's 0.625, at $0.60 versus $29.26 per 1,000 answers, about 49x lower. The edge model runs offline as a GGUF quantization on existing site CPUs, the cloud model is hosted via Bedrock Custom Model Import, and the whole pipeline uses seven AWS serverless services, going from plan to production in as little as six days.

NVIDIA Blog· Tutorials

C-BRAIN and NVIDIA Use Open Source AI Agents to Accelerate Alzheimer's Discovery

C-BRAIN is changing the pace of dementia research using AI, building a hybrid centralized and federated data processing system supported by NVIDIA, allowing AI models to learn from institutional "dark data" without compromising patient privacy. The research developed two tools: OpenScientist (an open-source agentic AI co-scientist) and InsightEngine (an open-source research platform). OpenScientist can complete analyses in minutes that domain scientists estimated would take weeks to months, at a cost of under $10 per run. C-BRAIN aims to discover new causal mechanisms of brain aging within three years by building a multimodal biological foundation model.

PickLatent Space· Tutorials

Coding Offline in the Himalayas With OpenCode and a Local Qwen3.5 9B

Living in the Himalayan foothills for a year, the author tested whether a coding agent could work with Wi-Fi fully off, running OpenCode against Qwen3.5 9B served locally via Ollama. She chose Qwen3.5 9B for its reported LiveCodeBench v6 score of 65.6 and BFCL-V4 score of 66.1, with a 6.6–7.6 GB download; the model, dependencies, and test tooling were all fetched before disconnecting, with Ollama started via OLLAMA_HOST=127.0.0.1:11434 and offline reachability confirmed with curl checks. In the offline run the agent fixed expense_total.py by changing total = Decimal(amount) to total += Decimal(amount), requested edit permission, ran tests, and the author independently ran python3 -m unittest -v to confirm all five tests passed with the program returning 25.50. She closes with practical ad…

The author ran OpenCode with a local Qwen3.5 9B through an actual offline bug fix in the Himalayan foothills, sharing the full setup, memory guidance, and reusable tips for small offline coding tasks.

Oct 27Tue1 pick · 6 stories

r/LocalLLaMA· Tutorials

Qwen3.8 27B on one RTX 5090: six inference engines benchmarked with and without DFlash2

The author benchmarked six inference engines (llama.cpp, vLLM, SGLang, NInfer, velox-inferer, TensorSharp) running Qwen3.8 27B on a single RTX 5090 32 GB, covering decode throughput, 32K prompts, concurrent requests, screenshot input, and a four-turn tool loop, with greedy decoding and thinking off. NInfer led speculative decoding and concurrent throughput at 488 tok/s, llama.cpp had the fastest plain decode at 84.2 tok/s, and TensorSharp reached 445 tok/s across four concurrent requests; NInfer posted the fastest 32K-prompt TTFT at 3.02 s. The author notes the engines used different quantized checkpoints and drafters, so results reflect complete configurations rather than runtime code alone, differences under ~5% are inconclusive, and velox failed the JSON-schema check in all three repet…

r/LocalLLaMA· Tutorials

Tool calling is still just a prompt: inside Qwen3's chat template and a gateway's tool loop

The author realized while building tool calling into their gateway that tool calling is still just prompt text. Qwen3's chat template pastes tool JSON into the system prompt between <tools> tags; the model writes a <tool_call> block as ordinary tokens, the server (vLLM, llama.cpp, or a hosted API) turns it into the tool_calls field, and results return as a user message wrapped in <tool_response> tags. Anthropic's docs similarly state that Claude Opus 5.5 adds a hidden 286-token system prompt for tool use before tool definitions. The gateway caps each tool result at 64 KB with a [cut: N more bytes] note, returns failed tools as Error: text instead of failing the request, and disables tools on the last round to force an answer; open source at https://github.com/respanai/respan-gateway.

r/LocalLLaMA· Tutorials

GPU FP32 accumulation unlocks explained: why Turing, Ampere and Ada can be unlocked but consumer Blackwell cannot

The author explains how recent GPU unlocks work: NVIDIA halves tensor core speed for FP16/BF16/FP8/TF32 with FP32 accumulation on consumer cards via factory fuses, and the unlock tools use a buffer overflow in the Booter firmware disclosed in Jon Pry's June 2026 paper to write FULL_SPEED to the override register from the highest privilege mode. Turing, Ampere and Ada are unlockable, with the 4090's FP16/BF16 FP32 accumulation going from 165 to 331 TFLOPS, 3090 Ti BF16 GEMM from 87 to 157 TFLOPS, and 2080 Ti from 47 to 94 TFLOPS; real-world GEMMs gain less than 2x due to power limits. Consumer Blackwell cannot be unlocked for now because it boots from an FSP (RISC-V root of trust) with no Booter to exploit and encrypted firmware. The author lists affected workloads (LoRA training, diffusio…

r/OpenAI· Tutorials

Reddit user built a home Star Trek-style computer, Ada, plus diagnostic panel BM-01, and shares a full postmortem

A user built Ada, an always-on conversational home AI, and just finished BM-01, an ESP32-based physical telemetry/diagnostic panel. The system runs on a Lenovo ThinkCentre M720q with GPT-Live realtime voice, a ReSpeaker XVF3800 four-mic array, local wake-word detection, persistent household memory, WiZ lighting and Samsung TV control, and a Red Alert emergency state. The author's main lesson is that the AI model isn't the hard part; infrastructure like the audio chain, barge-in handling, and memory relevance is. The writeup details migrating to a WebRTC/GPT-Live conversational core, a memory system that evolved into semantic embeddings with categorization, and testing a Microsoft custom keyword model in shadow mode against the existing ONNX wake detector.

PickAWS Machine Learning· Tutorials

AWS explains prompt caching patterns for OpenAI models on Amazon Bedrock

AWS engineer Claudio Mazzoni explains prompt caching patterns for OpenAI models like GPT-6 Astra on Amazon Bedrock, showing how caching the tool definitions, instructions, and history agents resend each turn cut input costs by about 62 percent in a six-turn workload. The post covers implicit and explicit caching, adding tools mid-session, warming subagents, and cache key design, noting a cacheable prefix needs at least 1,024 tokens with a minimum 30-minute TTL.

The post gives measured cost comparisons of explicit versus implicit caching on a six-turn agent workload, helping readers choose a caching strategy for their own agents.

AWS Machine Learning· Tutorials

Build agentic retrieval on a Managed Amazon Bedrock Knowledge Base with CDK

An AWS blog post shows how to build enterprise agentic retrieval on a Managed Amazon Bedrock Knowledge Base with AgentCore using AWS CDK: each knowledge base is exposed as an MCP tool through the Gateway, a reasoning agent handles multi-turn planning, semantic routing, and iterative retrieval, and the AgenticRetrieveStream API returns grounded, cited answers. Compared with the 2024 single-shot solution, no OpenSearch Serverless vector store or OCU costs are needed, and the deployment includes two CloudWatch dashboards covering seven telemetry layers plus on-demand and continuous evaluation. The whole system is one Python CDK app deployed with a single cdk deploy --all command across four stacks.

Oct 26Mon4 stories

r/LocalLLaMA· Tutorials

Multi-agent hedge fund postmortem: dropping our vector DB for SQLite BM25 and writing a 3.6ns Rust sentinel

The author rebuilt their open-source multi-agent hedge fund architecture after feedback from r/LocalLLaMA and shares three lessons. First, a Factor Ontology Schema with 11 quant factors pre-evaluates metrics deterministically as BULLISH, BEARISH, or RISK before agents see them, dropping polarity hallucinations in the investment council to 0%. Second, SQLite FTS5 BM25 replaced an Embedding API plus vector DB for reflection memory, cutting retrieval latency from 200-450ms to 0.35ms with better keyword relevance. Third, a real-time risk sentinel rewritten in pure Rust (SoA layout, `#![forbid(unsafe_code)]`) evaluates a single position stop in 3.63ns, 1,000 positions in 3.62 microseconds, and Python-to-Rust pipe IPC round-trips in 2.88 microseconds. The takeaway: LLMs handle contextual synthe…

TechCrunch · AI· Tutorials

Grok releases the first seven minutes of its AI-generated 'Odyssey'

Three months after Elon Musk announced a full-length, "historically accurate" Grok-made "Odyssey," Grok released a seven-minute "Episode 1" made with Wonder Studios and director Dev Walker. TechCrunch's reviewer found some shots passably realistic but the whole lacking artistry and momentum, a stark contrast to Nolan's practical-effects box-office hit.

r/LocalLLaMA· Tutorials

Tuning Strata 0.43: Qwen3.8-Flash-Next on RTX 4090 hits 45% faster decode

The author grid-tuned the Strata engine for an IQ3_S quant of Qwen3.8-Flash-Next on an RTX 4090 with 48 GB RAM under Windows, reaching 2076 tok/s prompt processing at 200k context and 95-105 tok/s decode with peaks of 145 tok/s. The three most effective settings were --adapt-async 1, --kv-resident 20480, and STRATA_ADAPT_LAG=2, combining for roughly 45% faster decode; they also found --expert-cache is in MiB not slots (72% slowdown when the card is filled), and Windows' inability to pin memory keeps expert copies CPU-driven. The author had Qwen3.8 tune itself and wrote a pi subagent extension that saves KV state, reaching about 450k total computed context.

r/OpenAI· Tutorials

The big question of 2026: Is a connector worth trusting with your email, files, and payments?

The author explains what you actually agree to when connecting an AI app: a connector is not a local program but a remote server exposing tools, involving three parties in the trust chain: the AI company, the connector operator, and the service holding your data. The main risks are threefold: OAuth permission scopes are often broader than needed, prompt injection means the AI can follow hidden instructions in text it reads, which is most dangerous when private data access, exposure to stranger text, and a way to send things out coincide, and the operator's legitimacy and data handling practices. The author proposes a permission ladder, starting from public data and read-only access up to send, pay or delete actions gated by human approval, illustrated with two open source MCP examples, Le…