What's New: Local AI is becoming more performant

(from the latest issue of the Indie Hackers newsletter)

"Private" local AI may not protect your data if your computer is compromised:

• Local AI shifts control from OpenAI, Microsoft, and Google to the people. Privacy is a strong selling point; build privacy-first, client-side apps.
• How do you bounce back from a failure or setback? Research shows that reconnecting with your core values can help. This list will get you started.
• $100K ARR with an AI SQL query tool. Mustafa Ergisi focused on product improvement based on continued, effective user feedback.

Don’t miss out on the Nike sale! Save up to 40% on select styles. #ad

Trend Alert: Local AI 🤖

from the Trends.vc newsletter

You pay for centralized AI tools that tell you what you can and cannot do, while saving your documents and innermost thoughts on their servers.

Local AI gives you more control over your data and usage. It becomes your advisor, not a supervisor.

Players

Local AI models:

• Llama 3: Local AI model from Meta.
• Gemma: Open source AI models from Google.
• Qwen2: Open source model.
• DeepSeek Coder: AI models that write, understand, and complete code.
• Stable Diffusion: Text-to-image generation model for photo realistic images.

Opportunities

• Build privacy-first, client-side apps. Privacy is a strong selling point for sensitive use cases.

• Build a user-friendly interface to help non-technical users connect, train, and use local AI. Great UI leads to great UX.

• Make tutorials to help people build, run, and use local AI models.

Predictions

• We'll see virtual companies of AI agents that work together locally.

• We'll be able to build AI apps visually, without code. This will let non-technical users build complex apps for their workflows.

• Lack of censorship will become a better selling point. Users will prefer unfiltered creativity over censored tools that refuse to do controversial, yet legal, tasks.

Risks

• Hardware requirements: If you're serious about running AI models locally, you may need to buy a new computer. Depending on your needs and preferences, this may cost a few thousand dollars.
• UX issues: You may not be able to use multiple models simultaneously. You can open ChatGPT, Claude, and Gemini in different tabs, but running more than one local AI model, with billions of parameters, can be impossible.

Key lessons

• The performance gap between local and cloud AI is closing.
• Local AI is self-sufficient. You can ask for help anytime, anywhere, as long as you have your device with you. No internet connection required.

Hot takes

• Governments will regulate local AI on par with centralized models. They still pose risks, similar to proprietary models.
• China will beat the US in the AI race. Chinese open source models already beat open source models from the US. Eventually, Chinese proprietary models will catch up, too. The US will try to limit public access to AI research. Such concerns have already been stated.

Haters

"I need more expensive, powerful hardware to run local AI models."

This is the main tradeoff for local AI at the moment, but it's becoming more performant. See how llama.cpp lets you run them on consumer devices, and how Apple is doing this on a grand scale. This may be an inflection point for hardware and local AI.

"Setup and onboarding is hard. I can't just visit a URL."

User experience with local AI is a solvable problem. We're getting there with open source tools that make setting up local AI easier. Ollama lets you set up Llama 3 in 10 minutes.

"Local AI models perform worse than AI models made by tech giants."

Open source AI models can be a little worse, but a lot more private and less censored. Depending on your use case, it can be wise to sacrifice quality for privacy.

"Providing support for models running locally sounds impossible."

This is another tradeoff of local LLMs. Unless the model becomes unusable, users can use an AI model to debug another AI model. This guy uses local AI models as copilots for coding copilots.

"Local AI models aren't a panacea for AI-related data privacy issues."

This is the risk of storing data in digital form. "Private" local AI may not protect your data if your computer is compromised.

Links

1. Open Source AI Is Wild.

2. Building a Report on Local AI.

3. Open LLM Leaderboard.

4. Why I Use Open Weights LLMs Locally.

Related reports

• Open Source AI.
• Niche AI Models.
• Data-as-a-Service.

Become a Trends Pro Member to get the full report on Local AI, or get the next free Trends.vc report here.

Discuss this story, or subscribe to Trends.vc for more.

Grab Your Favorites From Nike 👟

This issue is sponsored by Nike

Don’t miss out on the Nike.com sale! Save up to 40% on select styles, and upgrade your wardrobe with the latest gear. Whether you’re hitting the gym or the streets, now’s the perfect time to grab your favorites at unbeatable prices.

Hurry, these deals won’t last long!

Bouncing Back After a Failure 💪

by Dan Marzullo

Each week, the Indie Hackers editorial team tackles a reader question about the challenges of entrepreneurship. This week's question comes from Samuel N.:

How do I bounce back from a setback, and keep pushing forward to validate and sell my product?

Connect to core values

Reconnecting with your core values just might be the anchor you need to steer through these tough times and reignite your drive.

Your values are the things that matter most to you. When we face failures, setbacks, or challenging events in life and business, it's easy to lose sight of these values. Stress can take over, making us forget why we started in the first place.

Studies show that staying connected to our values makes us more resilient. Clarifying, articulating, and acting on these core values strengthens our ability to face adversity. In other words, our values give us a reason to keep going, especially when times are tough.

Getting back in touch

To reconnect with our values, we have to ask ourselves the deep questions. Often, we get so caught up in the daily grind that we forget to make choices that align with who we truly are.

Research suggests that even brief writing exercises can have powerful, lasting effects. They can help us tap into our values, and tackle challenging life events more effectively.

Dive into these questions to uncover what really matters to you:

• Why did you start this business in the first place?

• What would you like to get out of this business?

• Why is it worth getting through this slump?

• What will happen if you stop working on it and throw in the towel?

• What will your life look like if you make this product a success?

• What positive changes would it bring to you, and others?

• What key milestones or achievements do you believe are necessary for your business to be considered a success, leaving you with no (or few) regrets?

Write down as many answers to these questions as possible. Just let the words flow; capture whatever comes to mind.

Pull out your values

Take a look at the answers you've just written. What values capture the reasons why you want to keep going?

If you're feeling stuck, here's a list of values to help you figure out what's most important to you (courtesy of PositivePsychology.com):

• Achievement: Attainment of goals and aspirations.

• Fame: Recognition for your contributions.

• Adventure: Pursuing excitement and taking risks.

• Freedom: Independence in thought and lifestyle.

• Fellowship: Having important relationships with friends.

• Equality: Justice and fair treatment for all.

• Power: Authority, control, and influence.

• Service: Contribution to the welfare of others.

• Wealth: Influence, ease, and prosperity.

• Creativity: Being inventive, original, and innovative.

Keep your values top of mind

Write down your top three core values from this exercise on a sticky note. Place it on your desk or computer, where you can easily see it.

Whenever you're feeling unmotivated, take a moment to revisit those values, and remind yourself of what drives you.

And, if you've done all this and still can't connect the "why" behind your business to your values, it might be a sign of a different problem.

Got a question? Email me at [email protected]!

Discuss this story.

In the News 📰

from the Trendy Software Ideas newsletter

🖼️ Google's upgraded AI image generator is now available.

📈 10 of the best SaaS growth charts.

💲 Link to your product here. Our most affordable ad.

🤔 SEO vs. GEO.

📹 Shortform video is a powerful tool for lead gen.

🥑 The radical transformation of the avocado.

Check out Trendy Software Ideas to discover more software ideas inspired by recent news.

Early User Feedback Skyrocketed Growth 💬

by Mustafa Ergisi

Three years ago, I was a frustrated data analyst, spending countless hours optimizing SQL queries. The tedious process of manually tweaking queries for performance was driving me crazy. That's when it hit me: What if AI could do this automatically? With a background in machine learning and database management, I knew that I could build a solution. That's how AI2SQL was born!

Getting started

I decided to bootstrap. Looking back, it was the right choice for me.

Early user feedback was crucial. I shared the prototype on Product Hunt and a few SQL forums. The response was mixed, but the critics gave the most valuable feedback.

I iterated rapidly, focusing on the core feature of query optimization.

Growth strategies

Getting the first 10 paying customers was hard. I personally reached out to every free user, offering extended trials and gathering feedback. Content marketing became my main strategy. I wrote in-depth articles on SQL optimization, which started ranking well on Google.

Product Hunt and Hacker News launches gave good initial boosts, but the real sustainable growth came from SEO and word-of-mouth referrals.

I started with a simple $7 per month plan. It was too cheap! I realized that I was providing much more value than I was charging for, so I gradually increased prices and introduced tiered plans. Each price increase was nerve-wracking, but it actually increased conversions; people perceived more value.

Scaling to $100K ARR

The strategies that contributed most to reaching $100K ARR were:

1. Continuous product improvement based on user feedback.
2. Focused SEO efforts for long-tail SQL optimization keywords.
3. Strategic partnerships with complementary tools in the data space.
4. A referral program that incentivized word-of-mouth growth.

Lessons

What worked well:

• Focusing on a specific, painful problem in a growing market.
• Being highly responsive to user feedback.
• Content marketing for organic growth.

What I'd do differently:

• Hire help sooner, especially for customer support.
• Be more aggressive with pricing earlier.
• Invest in analytics from day one.

Advice for indie hackers

Talk to users constantly, focus on providing genuine value, and don't be afraid to charge what you're worth.

Embrace the challenges, celebrate the wins, and keep building!

Discuss this story.

The Tweetmaster's Pick 🐦

by Tweetmaster Flex

I post the tweets indie hackers share the most. Here's today's pick:

Enjoy This Newsletter? 🏁

Forward it to a friend, and let them know they can subscribe here.

Also, you can submit a section for us to include in a future newsletter.

Special thanks to Jay Avery for editing this issue, to Gabriella Federico for the illustrations, and to Dru Riley, Darko, Dan Marzullo, and Mustafa Ergisi for contributing posts. —Channing