Open Secrets: Using OSINT to navigate the online world

From eyewitness videos of police violence in Iran to satellite images showing the devastation of Gaza, an extraordinary amount of information is available to anyone with an internet connection. All that's required to source evidence is knowing where to look and how to use it effectively.

What is OSINT? 

Open-source intelligence, more commonly called OSINT, involves collecting and analyzing publicly available information. This information can include social media platforms, websites, online archives, public records like business registrations, satellite or flight data. Then it's about putting the puzzle pieces together to extract meaningful insights from the data you have collected. 

Open-source techniques are utilized in several professions like law enforcement to gather information on suspects or potential threats. The military uses them for strategic planning and journalists for fact-checking and investigations. 

How should I start? 

At the beginning of each open-source investigation, it's important to know what you are looking for and where to look for it. Do you want to find out where an image was taken, or who is behind a suspicious company, or if a source of information is reliable?

First, write down what you already know about the subject and what information is missing. This gap analysis, an OSINT methodology, enables you to systematically sort the information you have and identify missing pieces of your puzzle. 

Next, start searching for and gathering the information you need. This often means beginning with an internet search to get a first overview of the topic or subject. Subsequently, refine and narrow down your searches. 

Throughout the process make sure you confirm its accuracy. Verification is an essential part of open-source investigations. Any digital information can be made up. A video claiming to show the current conflict might have been filmed at an earlier date or filmed elsewhere. So, to check if it is real, verifying its provenance — where and when it was filmed is a crucial part of gathering information.

After gathering the information, the next step is to structure and organize it effectively. Connecting the dots and piecing together relevant information requires a good overview of your findings. In case you feel an information overload, which is normal in open-source investigations, organize your findings using tools like a mindmap, checklist or Excel sheet. Clear documentation is also useful for team collaboration, so everyone stays on the same page throughout the investigation.

[PICTURE 1. Copyright_DW Innovation] 

Additionally, save and archive your digital dataset in a centralized location as a backup. This will preserve the data for future reference for the safety of investigators and for presenting them in court testimonies or in media reports.

Once you have collected and organized your data, the next crucial step is to analyze and connect the dots to extract useful insights from it. For example, comparing before and after satellite imagery of the Russian invasion in Ukraine can give insights into the extent of damage to key infrastructure like hospitals.

To give another example where OSINT can be useful, if you interview eyewitnesses providing firsthand accounts of an incident, you can cross-reference their stories by gathering additional OSINT material from social media, such as images or videos, which may further corroborate their testimonies and provide additional context. This also enables the journalist to present a more complete and accurate account of what happened and might give insights into how the incident unfolded.

Having an investigatory mindset and working out different ways to solve a problem serves as a constant companion throughout the investigative process. Integral to this process is a critical analysis of sources and information.

The descruction of specific parts of Ukraine become evident by comparing satellite images of the same area in May 2022 and May 2023

What kind of information should I look for?

The amount of available open data on the internet is enormous. But simply viewing an image or accessing a website is not enough. The crucial part of open-source investigations is analyzing what you see and uncovering information that might not be immediately obvious.

When investigating a company's website, valuable information can be found not only in its visible content but also in its metadata and source code. Begin by using search operators in search engines. You can instruct Google on what you are searching for using these search operators. For instance, if you are seeking a specific filetype like pdfs on a specific website, you can combine the site search (which restricts a search to a single website) and filetype search (which restricts a search to a specific type of file) to more easily find this. For example, I could type into search: "OSINT site:dw.com filetype:pdf" to see what PDFs about OSINT are available on the DW website. 

Additionally, check reviews, search for current or former employees on LinkedIn, and use the Wayback Machine to track changes to the website over time. Investigate shareholders using NorthData and consult organizations such as OCCRP for additional information. Finally, analyze IP addresses to identify related web pages originating from the same source. 

There is so much more information that can be unearthed about any given subject. Here, we illustrate how to analyze an image and uncover more information beyond what is immediately apparent.

Always double-check your findings with a second source. As mentioned earlier, verifying your data is crucial. When you determine a data's source, also consider if it is legally obtained or part of a data breach?

If you are stuck, look for alternatives. For instance, Google cache is no longer available, meaning it won't save older versions of the website. So, use other search engines like Yandex and Bing or use the Wayback Machine. 

Just be aware, don't expose personal data. Handling sensitive information is part of any investigation. The focus should be on providing valuable insights without risking the exposure of private data.

What kind of OSINT tools I can use?

Many tools are available to help to access the information you require such as Google, map services like Mapillary, social media platforms like Instagram, Flightradar or the WayBack Machine for archived material, just to name a few.

Additionally, there are tools available to assist in collecting, organizing and analyzing the data you gather. For instance, web scraping tools that automatically extract data from multiple websites, mapping tools like QGIS that visualize information spatial data like radar satellite and AIS ship tracking data, or social media analysis tools that track trends and identify connections. Obsidian is a great tool to structure and visualize all your data.

Also bear in mind that some public information may be accessible but not for free. Some data may require payment to the service provider.

It's essential to use third-party tools and plugins you trust. Always consider the implications of uploading sensitive data such as an image to a reverse search engine, or a transcript of an interview with a whistleblower for translation. Understand how your shared data will be used.

If you need specific data, and you can't afford to pay for it or can't find an alternative source for it, reach out to the community.

Collaboration is key in OSINT. This collaboration may involve sharing insights, pooling resources, verifying information, and continuous learning with the OSINT community like here on Quiztime.

Just stay curious, be creative and share your knowledge.

AI tools can improve the manageability and effectiveness of AI investigations

What about AI and OSINT?

AI-supported tools present both challenges and opportunities in OSINT. Synthetic media like deepfakes or generative AI text-to-video like Sora from OpenAI challenges the authenticity and trustworthiness of digital content.

However, AI can also provide valuable support for open-source investigations. Try DorkGPT which generates google search operators for you or GeoSpy that helps to locate an image.

At DW's Research & Cooperation unit we use AI technology for location verification. As part of KID2, AI against disinformation, we're developing SPOT for accurate location verification. SPOT identifies geospatial patterns in OpenStreetMap. Users can input natural language queries, like "find a park in Perugia, Italy, with a viewpoint and a statue within 30 meters of a hotel bar starting with B." The app displays matching spots on a map.

So be prepared and train your senses, don't panic and open your browser to start digging.

This article is part of Tackling Disinformation: A Learning Guide produced by DW Akademie.

The Learning Guide includes explainers, videos and articles aimed at helping those already working in the field or directly impacted by the issues, such as media professionals, civil society actors, DW Akademie partners and experts.

It offers insights for evaluating media development activities and rethinking approaches to disinformation, alongside practical solutions and expert advice, with a focus on the Global South and Eastern Europe.