In addition to events discovered by RiskIQ's automated searching, users can also add their own events based on data from other sources and applications. This allows users to track this data within the RiskIQ system and take advantage of workflow, reporting, and other functionality within the platform to manage these items.
This article provides instructions for adding new events within the RiskIQ interface or API, including:
- Direct event submission for pre-confirmed data (create an event of a specified type without analyzing against policy logic)
- Crawl submission (submit an unconfirmed URL to get analyzed and potentially make an event of the appropriate type--or no event if it does not qualify based on RiskIQ's analysis of the page against your configured policies)
- Diagram of event creation paths and system workflow
Events are notifications within the RiskIQ system that provide information about a particular type of threat. Which event-types you see in your workspace will depend on the RiskIQ products you have purchased and your access permissions as a user. Talk to your RiskIQ Technical Account Manager if you have any questions about the events enabled in your workspace or about user permission controls.
Your events can be viewed in app.riskiq.net by selecting 'Events' from the navigation menu at the top right, or by clicking on any of the graphics in the Events Dashboard.
Direct Event Submission
Add Events in the UI
To submit events on an ad hoc basis in the UI, use the plus-sign icon in the top right corner of the event screen (next to the export button, page settings, user help, and profile settings).
This will bring up a dialog where you can choose the type of event you want to create and enter details, including the grain of the event (valid entries will depend on the type you are adding), and specify any other metadata about the event, including tags, status values, or notes. See Event User Actions for more details on status values and other workflow actions.
Submitted events must be a unique combination of grain and event-type--if an event already exists for the same data entered, you will receive an error message and a duplicate event will not be created.
The types of events you can select from the dropdown list are limited to those that are enabled in your workspace (based on the products your organization subscribes to), as well as your personal user permissions (user must have edit access for the event-type--read-only users cannot submit new events).
After you add the event, the dialog will close. When your event has been created, a notification will appear in the top right of the screen with the event ID. Click on the event ID link to view the new event without losing your current applied filters, or else, you may simply close the notification by clicking on the X.
The user who created the event is shown in the history of the event along with any updates specified in the dialog at the time of creation.
Additionally, the source of your event will show up as "User Submission" (or "API Submission" if submitted through the API rather than UI), so that you can filter for events submitted in this fashion vs. those found by RiskIQ through other means using the Source field. Note that direct submission is the only possible source for creating Custom Events.
For event-types that include crawling a URL, a crawl will start upon your event submission. Some details in the event that are based on the content of the page (including screenshots and classifier attributes) may not appear in the new event immediately, but will populate when the crawl is completed and processed.
Once created, your event will enter the standard system workflow and monitoring schedule. This means that your events will continue to get re-crawled / re-checked over time just like events of the same type created by RiskIQ. Enforcement, auto-resolution, monitor status, and tenacious behavior is also all the same.
Thus, while you may be able to manually add events that wouldn't otherwise trigger your policy if crawled by RiskIQ, those events will automatically resolve over time unless adjustments are made to your policy configuration to capture all the relevant data for your use case.
Add Events via API
Events can also be added programmatically using the RiskIQ API. This method is best for adding new events in bulk vs. one-by-one, and can be used for integrating data from other applications that do not fit into a RiskIQ event-type as custom events (crawl submission can be used for integrating data from other systems that does related to a supported event-type--see below).
A valid API token and password is required to use the RiskIQ API. Ask your RiskIQ Technical Account Manager if you do not have or do not know your API credentials. See RiskIQ API Documentation for more details.
Requests consist of just a valid grain input for the specified event-type.
Required Fields by Event-Type
- Phish: URL (ex. "http://www.site.com/page/")
- Domain Infringement: Fully Qualified Domain (ex. "domain.com" or "subdomain.domain.com")
- Rogue Mobile App: App URL (ex. "http://appstore.com/app/")
- Content: URL (ex. "http://www.site.com/page/")
- Social: Profile URL (ex. "https://socialnetwork.com/user/username")
- Custom: Title (ex. "(510) 444-4187"), Description (ex. "This phone number is part of an investigation into....")
*Note that if your input is not of the format listed above, you will not be able to create an event. For Social and Rogue Mobile App events, this means that the app store or social network must be recognized by the RiskIQ system as valid.
To create new events within the RiskIQ platform with data that is not pre-analyzed or confirmed (may contain false positives mixed in with good data), you can submit URLs for RiskIQ to crawl and analyze against your policy as candidates to create events rather than submitting the events directly.
Common use-cases for this method include using referrer logs and proxy / web server logs, DMARC forensic reports, or abuse boxes as additional data feeds for phish or other types of events. Refer to those articles for more details on each type of integration.
Crawl submission is currently only supported via the RiskIQ API (requires a valid API token and password) or via email.
Landing Page API Submission Request
A valid URL is required for submission, which can be up to 2048 characters. Other parameters may optionally be set as well to govern the crawl settings. The most notable of those settings include:
- Keyword: Can be used to generate a referrer for this landing page if desired
- Optional MD5 representing the canonical ID for this landing page: If none is specified, then one will be generated based on the batch, URL, and keyword (The MD5 must have an even number of characters, be at most 32 characters in length, and include only hexadecimal characters; case insensitive)
- Project Name: Sets the source label of the event (useful for reporting and filtering; also controls crawl behavior, in particular what geolocation the URL will be crawled from, and what browser-type will be used); available project names may be fetched via /landingPage/projects/
- Field: Add custom metadata with this submission (e.g. an ID number from another application) to associate with this crawl
Example: Landing page submission with URL and MD5 specified
Example: Landing page submission with URL, MD5, and 2 custom fields (source and sourceID) specified
Landing Page Retrieval Methods and Response
Once submitted, landing pages will be crawled, and those crawls will be analyzed against policies configured in your workspace to create events. You can view the results in the UI based on the events that get created, or you can look up the crawl(s) via the API (to see its details whether or not an event got created).
There are 3 ways to retrieve landing page results:
- GET / Bulk GET: The most direct way to retrieve results is to look up a landing page crawl by MD5
- Retrieve by Time: Specify optional start and/or end timestamps to retrieve all LPs crawled after, before, or during a certain time.
- Ping Back: Get a real-time notification whenever your crawls are ready for retrieval by specifying a "ping back" URL where a http GET request will be pushed as soon as the crawl has completed processing. The "ping back" request itself doesn’t contain the info, but it tells you when you are able to do a GET request on the particular MD5.
Details returned for each landing page result include all the metadata about the submission and the ability to pull down the DOM of all pages crawled, the sequence and final URL/IP if there was a redirect chain, a link to view the screenshot of the final page, classifier scores, and crawl inspection results (includes time of inspection, whether an event was created / Type / ID, there is a pre-existing event associated to this URL / Type / ID, or no event was created). There are 2 inspection pipelines (phish and normal / everything else), and at minimum for any given crawl, there will be a blank entry for both when inspection is complete. There will be at least one non-blank entry in the inspection results if any events are associated, making the total number of entries at least 3.
For example: a crawl monitoring an existing phish event shows the ID of the event, but says event created is false to indicate that it is an existing vs. a new event ID. Below it are blank entries for both the phish and normal pipelines indicating the inspection of each type is complete and no more events will be created or updated with this crawl. Blank entries are pruned and deleted after 30 days.
For each event ID, you can look up more details about the event, such as it's current status, first, most recent, and next scheduled crawls, and enforcement information related to the event. See Event Search for more information on using the API to view events. You can also make changes to events including changing status or adding tags. See Event User Actions for more details.
Note that you can also submit items to crawl via email instead of API. Typically this is done through an Abuse Box Integration wherein an email alias within an organization (ex. firstname.lastname@example.org, email@example.com, etc.) is set up to auto-forward all messages to a dedicated email address RiskIQ creates. You can also send one-off emails to this address in order to initiate a crawl automatically on demand.
When an email is received, RiskIQ strips out all links found in the message body or attachments and submits them for crawling. Each email address is associated to a project whose settings define what geolocation and browser type(s) should be used to crawl submissions from that source.
Event Creation System Overview Diagram