BTS of a Competitor Spying Tool I Built

👋 Henry here, I received an overwhelming amount of positive replies to last week’s article where I shared a BTS of a tool we built for a consulting client (thank you!!!) so thought I would share another one. Insights from today’s article:

• 🗺️ A tool we built to spy on competitors

• 📊 How to design UI for AI-products

• 🕵️ How to reduce AI hallucinations

If you have any other ideas for tools just send me a reply.

Context

Competitor intelligence has been something I’ve been a student of for about 4 years. To this day I think it’s one of the most valuable parts of pricing and CRO design and something very few others do.

If you can track and effectively distill what your competitors are doing you have unfair advantages, it’s as simple as that. So few early-mid stage companies have the capabilities.

If you know which messaging is resonating, or you’re tracking successful A/B tests, or even designing your pricing to counterposition against competitors then you can grow faster than anyone else. You don’t need to stumble around in the blind, just copy what’s working for others.

What’s the problem?

Today B2B SaaS is ridiculously competitive. From the stats I’ve seen competition is up 15x, and for certain categories like AI it may even be more. Larger incumbents are being incredibly aggressive to launch new products & features, we’re seeing $20m seed rounds and the amount of AI SDRs I’ve seen boggles the mind.

As a result every company right now is trying to balance the fact that most enterprises are still on slower spending, whilst needing crazy growth to match competition and satisfy investors.

But for our client it was a little tough to figure out how to position their offering:

• Positioning: Are they creating a new market or positioning it as a remarkably better version of an existing tool so enterprises are already looking to buy their solution? E.g. copilot vs agent

• Pricing: Do they charge based on outcomes? Based on usage? A flat SaaS fee? Are they pricing in a similar way to competitors?

• New Features…

They were really struggling to get these right - were ready to heavily invest in paid acquisition but the current unit economics just didn’t support that. Their payback periods for the PLG product were 13 months and only had 8% inbound, the vast majority was driven by cold email and paid ads.

What was the solution?

We took a crack at a custom built GTM software solution, that would scrape their competitors' sites - home pages, pricing etc and even their Facebook ad library, and parse all the information so it could create alerts for any changes and help the client figure out what GTM motions were working.

The tool used OpenAI’s Vision model to take screenshots of the pages, and compare it against previous versions to determine whether there were any meaningful differences.

For some major ones like pricing changes it would immediately show alerts, and for things like changes in homepage copy it would track the change and if it lasted longer than 2 weeks it was a pretty good signal the competitor’s test was successful.

But because AI has a tendency to ‘hallucinate’, especially with vision models it’s far too error prone to deploy into production for most use cases.

This means we had to design the scraper so that users could scan the top level info and then dig in and verify the information was correct, for any interesting actions.

So what?

(1) Allowed us to find messaging angles that really worked. How you market a product is rapidly changing in the world of AI and it’s very important to know if competitors have announced new features. We found ads from a competitor that had been working for the past 6 months and adjusted accordingly.

One great example of this is IronClad. For years they tried to create a new category (AI Legal Assistant) which was a total slog requiring you educate users on the need. They changed their positioning to play in an existing category (Contract Management System) where customers were already looking to buy, and growth took off.

(2) Allowed us to track pricing across the industry and built pricing that counter-positioned against competitors. Most were seeing one key feature as their core value prop and so were putting this feature in a higher tier to get more people upgrading.

We made this feature free and charged for other features which made us far more attractive to users and easier to demo, and competitors struggled to adapt the same pricing because it was the main revenue driver.

Hope this was helpful!