Data Leak - Core Algorithm

Watcher.data_leak.core.check_keywords(keywords)

Check keywords in Searx Instance & Pastebin.

Parameters:

keywords – Keywords stored in database.

Watcher.data_leak.core.check_pastebin(keywords)

Check Pastebin for keyword list.

Parameters:

keywords – Keywords stored in database.

Returns:

Matched urls & Corresponding keyword.

Return type:

dictionary

Watcher.data_leak.core.check_searx(keyword)

Pull Searx instance for keyword.

Parameters:

keyword – Keyword stored in database.

Returns:

Matched urls.

Return type:

list

Watcher.data_leak.core.check_urls(keyword, urls)

Check if the URL is new.

Parameters:

• keyword – Keyword stored in database.

• urls – Fresh searx urls.

Returns:

Urls not already in alert database column.

Return type:

list

Watcher.data_leak.core.cleanup()

Remove 2 hours old, useless, pasteIDs.

Watcher.data_leak.core.main_data_leak()

Main function:

• close_old_connections()

• read in our list of keywords

• check_keywords(keywords)

Watcher.data_leak.core.send_email(alert)

Send e-mail alert.

Parameters:

alert – Alert object.

Watcher.data_leak.core.send_group_email(keyword, alerts_number)

Send group e-mail for a specific keyword.

Parameters:

• keyword – Matched Keyword.

• alerts_number – Number of alerts.

Watcher.data_leak.core.start_scheduler()

Launch multiple planning tasks in background:

• Fire main every 5 minutes from Monday to Sunday

• Fire cleanup every 2 hours