You are here: Home / Competitive intelligence tools and techniques

Competitive intelligence tools and techniques

While this section on competitive intelligence tools and techniques often shows how to reverse engineer companies engaged in questionable actitivies – the truth is that these same investigative tools and techniques can be used for MANY different things including:

  • Spying on marketers and determining what niches they are in and what affiliate offers they are promoting.
  • Finding multiple companies operated by the same organization or organizations.
  • Collecting URLs for various types of marketing (although I’m not a big fan of PPV, many of these tools and techniques would be quite useful for that type of marketing).
  • Finding potential JV partners.
  • Identifying potentially profitable sales techniques.
  • Reverse engineering social networks and analyzing the strength of connections within those networks.

I’ll add more as I think of them, in the meantime here are the current posts on competitive intelligence tools and techniques broken down by the type of entity being analyzed.

Competitive Intelligence Techniques

Websites:

Companies and People

Images & Videos

Buzz

Competitive Intelligence Tools

Automation Tools

  • Automator Virtual Input (Apple) – Extends Apple’s Automator automation program to give it control over the keyboard and mouse cursor. Automator Virtual Input allows Automator to access anything you can by simulating mouse clicks, movement and drags, etc.
  • IOpus – web automation, testing & extraction tool.
  • mjtnet – Windows automation software
  • ScrapeBox – Powerful URL harvesting, analysis, and management tool
  • Web Data Parser – an inexpensive and pretty effective web data extraction tool for a lot of purposes
  • Ubot Studio – extremely robust automation tool for developing “bots” through a visual scripting language
  • URL Detective – Whitehat URL harvester that uses the Google API
  • WinAutomation - Windows Automation Software

“Swiss Army Knife” type tools:

  • Maltego – A short demo of Maltego (extremely powerful investigation tool  with a free version). “Maltego is an open source intelligence and forensics application. It will offer you timous mining and gathering of information as well as the representation of this information in a easy to understand format.”
  • Outwit Hub – “An all-in-one application for extracting and organizing data from online sources; a powerful automation engine that explores for you large series of Web sources and brings back the answers instead of telling you where they are.”
  • DomainTools.com (especially the paid version) – incredibly powerful tool for exploring almost anything you can think of when it comes to domains. Probably the most important tool I use.
  • Builtwith.com – determines site technology used as well as types of analytics, etc.
  • DomainCrawler.com – DomainCrawler 2.0 includes extensive functionality such as thumbnails, backlink checks, pagerank history and the ability to monitor a domain for change as well as more traditional domain/SEO-related information.
  • DNSStuff – Extensive collection of DNS analysis, tracking, and alert tools.
  • IP-Address.com – Several IP tools, including the most accurate IP locator tool (other than MaxMind.com’s bulk GEO IP locator) I’ve found yet.
  • MarkosWeb.com – Uses around 25+ open API sources researching the mo st popular domains and keywords. Helps webmasters check their competitors and understand why their ranks are so high and includes useful summary tables for webmasters.
  • Network Tools – Traceroute, Ping, Domain Name Server Lookup, Whois, and more
  • Push To Check – offers a service that checks different parameters for domains and subdomains and saves their history. So you can return again later and check for progress of your sites or sites of your competitors.
  • QuarkBase.com – See people, traffic data, similar sites, social comments, description, social popularity and much more about website
  • Robtex.com – Swiss Army Knife internet tool – network exploration, blacklist checking, and a lot more
  • SensePost Tools – Numerous Paid and Free Tools
  • Serverniff.net (NOT .com) Your free “Swiss Army Knife” for networking, serverchecks and routing with many many little toys and tools for administrators, webmasters, developers, powerusers und security-aware users.
  • Siteopsys – 30 research and analytical SEO tools. One nice tool is the bulk reverse-ip look up tool. Not as accurate as using WebDataParser against DomainTools, but a nice quick and dirty way to find potential groups of sites on IPs to explore further.

Tools developed specifically to determine potential ownership between websites

Tools to pinpoint potential footprints to run through Footprint Finding Tools

Of course almost any of the tools on this page can be used to pinpoint potential footprints. But the following tools are great tools to run against the results of the OTHER tools on this page to more easily pinpoint potential footprints to run through Footprint Finding Tool

  • Ghostery – A privacy tool which can be used to determine what types of analytics, advertising, and tracking codes a site is using.
  • Maltego – Extremely powerful investigation tool. A great way to find potential footprints you may have never considered
  • ServerSiders.com – has a section which sometimes shows if a site is running Google Analytics or Google Adsense and provides the IDs. Not always accurate and is best use to feed the Ids through the tools developed specifically to determine potential ownership between websites.
  • OutWit Hub – “An all-in-one application for extracting and organizing data from online sources; a powerful automation engine that explores for you large series of Web sources and brings back the answers instead of telling you where they are.”

Footprint Finding Tools

In a sense most of the tools on this entire page could be used as “Footprint Finding Tools” – but these are Footprint Finding Tools I find myself using time after time.

  • CopyScape – Find duplicate versions of content
  • ScrapeBox – Powerful URL harvesting, analysis, and management tool
  • Tineye – Currently the best reverse-image search engine online

Tools To Determine Uniqueness Of A Footprint

Keyword Research And Tracking Tools (Spy on competitors keywords, ads, campaigns, and more)

  • Keyword Spy (Paid Version) -  Uncover the competition landscape by knowing your competitors’ PPC and Organic campaign strategies in seconds.
  • PPC Web Spy (Paid Version) – PPC Web Spy allows you to secretly spy on & uncover anyones Google-Adwords keywords while you browse Google in real time.
  • iSpionage (Paid Version) – Discover competitors’ top keyword, ad copy, and landing page strategies
  • SEMRush – SEMRush shows Google keywords for any site, Adwords keywords for any site, list of competitors in organic and paid Google search results, & estimated SE and Adwords traffic for any domain
  • SpyFu (Paid Version) SpyFu allows you to get information on the competition, including how they are using keywords in paid advertising and organic search as well as other information on their advertising strategies.
  • PPC Bully (Paid Version)

Other Traffic Analysis Tools

  • Alexa – “Swiss Army Knife” like collection of traffic analysis/competitive intelligence tools – Alexa has become MUCH more useful than it has been in the past. And it’s free.
  • Compete – Another “Swiss Army Knife” collection of traffic analysis/competitive intelligence tools. The best data from Compete is not free.
  • Quantcast – Provides detailed audience profiles for the advertising marketplace
  • UpDowner – Fastest growing and dropping websites by search, bounce-rates, page views, and more. Can search via keywords.
  • Web Site Traffic Spy – A traffic aggregator using Alexa, Compete, Quantcast, Google Trends, and the IP location information from IP-Address.com

Proxy Detection Tools

SSL Tools

Website/Web Page Change Detection Tools

  • Archive.org - Archival history of websites and their pages
  • ChangeAlarm.com – free website change detection tool (free use subject to Terms of Service)
  • ChangeDetection.com – free service provides page change monitoring and notification services to internet users worldwide. Anyone can use our service to monitor any website page for changes. Free since 1999 (free use subject to Terms of Service)
  • URLy Warning – free, easy-to-use Windows software that lets you monitor Web pages. It works great with any XP, Vista or Windows 2000 PC. Know instantly when a page changes. See exactly what’s changed on that page.
  • WatchThatPage.com – a service that enables you to automatically collect new information from your favorite pages on the Internet. You select which pages to monitor, and WatchThatPage will find which pages have changed, and collect all the new content for you. The new information is presented to you in an email and/or a personal web page. You can specify when the changes will be collected, so they are fresh when you want to read them. The service is free!* *Professional and heavy users are requested to pay a fee. See terms & conditions

Website Exploration Tools

  • Fox Splitter – splits the content area of the browser window almost any way you would want.
  • HTTrack Website Copier – Offline Brower
  • OutWitHub.com – “An all-in-one application for extracting and organizing data from online sources; a powerful automation engine that explores for you large series of Web sources and brings back the answers instead of telling you where they are.”

More Specialized Utilities/SitesTools

  • Ip2Location.com – IP2Location™ is a geo IP solution to help you to identify visitor’s geographical location. Highly accurate.
  • Bulk GEO IP locator at MaxMind.com – Find countries and cities by entering IP addresses/hostnames/domains
  • Domain Status Reporter – Obtain and monitor the registry status (active, registrar-lock, registrar-hold, etc), created, updated and expiry dates of global top level domains (com, Net, org, Edu, Biz and Info).

URL Decoders:

Competitive Intelligence – Media Monitoring Services

  • LiveDash – Search What’s Being Mentioned On T.V.
  • Publisher’s Information Bureau – check on the competition by seeing what publications they are using repeatedly.
  • Experian Simmons – extensive consumer studies  report over 60,000 data variables in over 8,000 product categories and 450 plus brands, major media usage behavior, and in-depth consumer behavior demographics to give our clients the ability to make the most of their marketing budget.
  • Whos Mailing What - Direct Mail Competitive Intelligence Archive

Buzz Tools

News & Media Analysis Tools

  • Recorded Future – Recorded Future’s linguistics and statistics algorithms extract time-related information and through temporal reasoning helps users understand relationships between entities and events over time, to form the world’s first temporal analytics engine
  • SiloBreaker – aggregates content on global issues, science, technology, energy, business, sports and entertainment from tens of thousands of news sources, blogs, multimedia, and other forms of news media from around the world. With the engine’s focus on finding and connecting related data in the information flow, Silobreaker’s user tools and visualizations are ideal for bringing meaning to content from either today’s Web or the evolving Semantic Web, or both.

Forum Buzz & Reputation Tools

  • Omgili Graphs – Measure and compare the Buzz of any term. The Buzz is the percentage of the term out of the total number of discussions Omgili covered on a specific date.
  • Boardreader – Forum search engine
  • BoardTracker (Premium version)  – Social media metrics with detailed graphs and advanced analytics based on quality data.

Social Media Buzz, Analysis, & Reputation Tools

  • 123People.com – (Note: not as good as Pipl) -  free people search
  • BrandsEye.com – Inexpensive reputation management tool
  • BuzzDing – Monitor your reputation online and connect with the community surrounding you, your company, and your products
  • HowSociable? – Type in a brand name, keyword, website, etc.. Find out how visible it is on the social webs.
  • KnowEm – Check a company’s or person’s profile presence across 400 social media networks
  • PeekYou – asdf
  • Pipl -  “People search” deep web search engine. Can also search by email, username, phone number, and business name
  • ScoutLabs – Track, Manage, and Analyze discussions across social media platforms. Includes sentiment analysis engine.
  • socialmention – Real-time social media search and analysis
  • SocialWhois – Type a nickname and select a service (Twitter or FriendFeed) and SocialWhois will try to give you some information about the company or person associated with that nickname.
  • Spokeo – Spokeo is a search engine specialized in organizing people-related information from phone books, social networks, marketing lists, business sites, and other public sources
  • Trackur – Suite of social media monitoring tools
  • Trendistic – see trends in Twitter
  • Whos Talkin – Social media search tool
  • Yasni.com – Find a person is mentioned on the web

Social Network Diagraming Tools

  • Gephi – an interactive visualization and exploration platform for all kinds of networks and complex systems, dynamic and hierarchical graphs.
  • NetDraw
  • NodeXL – Free Network Overview, Discovery and Exploration Template for Excel
  • Pajek - Windows program for analysis and visualization of large networks – freely available, for noncommercial use.
  • SocioGram - A simple program to generate sociogram-graphics showing the relationships between individuals
  • Software list at INSNA (obviously I haven’t tried all of these)

Image Analysis Tools

  • Tineye – Currently the best reverse-image search engine online
  • Focus Magic – uses advanced forensic strength deconvolution technology to literally “undo” blur and recover lost detail.  It can repair both out-of-focus blur and motion blur (camera shake) in an image. Extremely inexpensive considering how powerful it is.
  • DeblurMyImage - Deblurring software for out of focus correction and motion correction
  • DenoiseMyImage – allows you to remove digital noise from your photos while preserving edge.
  • ABBYY Tools – Especially the FineReader OCR solution for extracting text from images.

Text Manipulation Tools

  • Clean Haven – Mac, OS X, Windows, and Linux based tool for cleaning text
  • Convert Case – Simple online tool to convert case of characters
  • Data Mystic – Data Extraction, Data Conversion, Report Mining and Text Manipulation Software
  • SortMyList – Online tool with a large variety of ways to sort lists
  • Text_Diff – Show differences between 2 works of text
  • Text Mechanic – An extensive collection of free, online, browser-based, text manipulation tools. Includes harder to find tools such as text randomizing tools, text obfuscation tools, permutation & combination generators and more. Also offers a basic online html WYSIWYG editor.
  • Text Mechanic – An extensive collection of free, online, browser-based, text manipulation tools. Includes harder to find tools such as text randomizing tools, text obfuscation tools, permutation & combination generators and more. Also offers a basic online html WYSIWYG editor.

If You're Struggling To Make Money Online - Click Here To Watch This Free Video And FINALLY Get Answers To All Of Your Questions About Making Money Online

TAG AT THE END OF YOUR WEB PAGE -->