Data mining, darknet and chat monitoring – a deep dive into Telegram monitoring and the latest features of the AIL framework

Wednesday 4 October 16:00 - 17:00, Small Talks room

Steve Clement (Luxembourg House of Cybersecurity) & Aurélien Thirion (CIRCL)

Data mining, darknet, chat and social network monitoring are critical components of modern threat intelligence and security operations. The AIL Project is an open-source framework designed to collect, crawl, dig, and analyse unstructured data from various sources. With its extensible Python-based framework, AIL can analyse unstructured data collected via an advanced Crawler manager or from different feeders such as PasteBin-alike sites, Twitter, Discord, Telegram Stream providers, or custom feeders.

AIL supports active crawling of Tor hidden services, protected websites and forums with pre-recorded session cookies. Its modular design allows for easy contribution and extension, enabling the addition of new analyser modules, feeders, or streams without the need to know the inner workings. The framework also has integrations with other open-source projects such as MISP or cve-search. AIL project is a nice companion with other OSINT tooling to support intelligence analysis or proactive monitoring or gathering insights from different topics.

This presentation will focus on the latest version of the AIL framework (v5.3) and its new features such as chat monitoring. Attendees will learn about the following:

• An overview of the modular design of AIL and its extensibility through analyser modules, feeders, and exporters.
• Demonstrations of how the new features can be used in practice, including practical examples of investigating Tor hidden services and other sources of data.
• Best practices for data collection and analysis with AIL, including performance optimization techniques and integration with other open-source projects such as MISP.

The audience will learn how AIL can help you with your data analysis and threat detection needs. The goal is also to gather new ideas to improve the open-source tools and create new collection/analysis concepts.

• Download slides for this talk

Steve Clements Meet Steve Clement, a security veteran at Luxembourg's House of CyberSecurity (LHC) since 2008. Specializing in Unix system security, he's dedicated to sharing knowledge in the infosec community. Steve tracks global cyber trends, focused on enhancing threat sharing in our interconnected world. With over 20 years of tech, start-up, and hacking experience, he champions free and open-source software/hardware to dismantle barriers. At LHC, Steve aids private sector incidents and supports cybercrime victims. He underscores the vital role of efficient indicator of compromise (IoC) sharing in robust IT strategies. Collaborating with experts, he works to streamline secure data exchange for a safer digital future. @SteveClement  @SteveClement

Aurélien Thirion Meet Aurélien Thirion, an open-source developer and security researcher since 2020, specializing in leak detection and secure information sharing. He's a core developer of the AIL project at CIRCL and contributes to the MISP project, all while maintaining tools like Passive SSH. His commitment to innovation and collaborative progress defines his impactful career in open-source development and security research. @terrtia  @terrtia

Back to VB2023 Programme page

Back to VB2023 conference page

Register for VB2023

Other VB2023 papers