Data Extraction Commands

Data extraction commands are the heart of any scraping automation. They tell RTILA X what data to collect from the pages you visit and where to store the results. Without extraction commands, your automation would visit pages but never save anything useful.

RTILA X provides five data extraction commands, each serving a different purpose:

Command	Purpose
`extract_data`	Runs a full dataset extraction job and saves results
`get_text`	Gets the visible text of a single element
`get_attribute`	Gets an HTML attribute value (like `href` or `src`)
`get_property`	Gets a JavaScript DOM property (like `value` or `checked`)
`is_visible`	Checks whether an element is currently visible

extract_data

The extract_data command is the most powerful extraction command. It runs a complete dataset extraction job using a dataset configuration you’ve defined in the Visual Builder, and saves the results to a file, the API, or a variable.

Description

When RTILA X executes extract_data, it:

Looks up the dataset configuration by name from your project.
Finds all elements matching the dataset’s item selector on the current page.
For each matched element, extracts all defined properties.
Applies any transformations you’ve configured on each property.
Checks required fields and logs warnings for missing values.
Deduplicates results against previously extracted data (using checkpoints).
Saves the results to the configured output destination.

Parameters

Parameter	Type	Required	Default	Description
`dataset`	string	✅ Yes	`""`	The name of the dataset to extract. Must match a dataset defined in your project’s dataset configuration.
`output_filename`	string	No	`""`	The file path where extracted data will be saved. If empty, data is saved via the API (if connected) or to the default output location. Supports `${variable}` substitution.
`append`	boolean	No	`true`	If `true`, new data is appended to an existing output file. If `false`, the file is overwritten. Only applies to file-based output.
`parent_data_from`	string	No	`""`	The name of a variable containing parent data to merge with extracted results. Useful for combining data from multiple extraction steps.
`output_variable`	string	No	`""`	If set, the extracted data is stored in this runtime variable instead of (or in addition to) being saved to a file.
`filter_condition`	string	No	`""`	A JavaScript expression that filters extracted items. Items where the expression returns `false` are excluded. The current item is available as `item`.

Examples

Basic extraction:

{
  "command": "extract_data",
  "params": {
    "dataset": "products"
  }
}

This extracts data using the “products” dataset and saves it via the default output method.

Extract to a specific file:

{
  "command": "extract_data",
  "params": {
    "dataset": "products",
    "output_filename": "output/products_${date}.json",
    "append": false
  }
}

Extract with a filter:

{
  "command": "extract_data",
  "params": {
    "dataset": "products",
    "filter_condition": "item.price > 10 && item.title.includes('Wireless')"
  }
}

Only items where the price is greater than 10 AND the title contains “Wireless” will be saved.

Store results in a variable for later use:

{
  "command": "extract_data",
  "params": {
    "dataset": "products",
    "output_variable": "product_list"
  }
}

The extracted data is stored in the product_list variable, which you can then use in a for_each loop or other commands.

Merge with parent data:

{
  "command": "extract_data",
  "params": {
    "dataset": "product_details",
    "parent_data_from": "parent_product"
  }
}

This merges the extracted product details with data stored in the parent_product variable.

Behavior Notes

Deduplication: RTILA X automatically deduplicates extracted data. Each extracted item is hashed and compared against previously extracted items in the current run. Duplicates are skipped. This prevents duplicate records when scraping paginated content.
Required fields: If your dataset has required fields configured, RTILA X logs a warning for any item where a required field is empty, but the item is still included in the results.
File output: When output_filename is set, data is saved to that file. The format is determined by the file extension (.json, .csv, .xlsx, .parquet). If the file already exists and append is true, new data is appended.
API output: If no output_filename is set and you’re connected to the RTILA API, data is saved via the API to the project’s dataset collection.
Standalone mode: In standalone bot mode, if no output_filename is set, data is saved to a CSV file in the working directory.
Transformations: Any transformations configured on dataset properties are applied during extraction. See Transformations for details.
Variable selectors: If a property uses a variable= selector, the value is read from the runtime variable store. This allows you to inject dynamic values into extracted data.
Download properties: If a property has the download option enabled and the extracted value is a URL, RTILA X downloads the file and stores the local file path instead of the URL.

get_text

The get_text command gets the visible text content of a single element and stores it in a runtime variable.

Description

Use get_text when you need to capture a single piece of text from the page — like a page title, a price, a status message, or a confirmation message. The text is stored in a variable that you can use in later commands.

Parameters

Parameter	Type	Required	Default	Description
`selector`	string	✅ Yes	`""`	The CSS, XPath, or text selector for the element. Format: `css=...`, `xpath=...`, or `text=...`.
`output_variable`	string	No	`""`	The name of the variable to store the text in. If empty, the result is returned but not stored.
`iframe`	string	No	`""`	If the element is inside an iframe, provide the iframe’s selector here.
`timeout`	number	No	`5000`	Maximum time in milliseconds to wait for the element to appear.

Examples

Get a page title:

{
  "command": "get_text",
  "params": {
    "selector": "css=h1.page-title",
    "output_variable": "page_title"
  }
}

Get text from an iframe:

{
  "command": "get_text",
  "params": {
    "selector": "css=.status-message",
    "iframe": "css=iframe#content-frame",
    "output_variable": "status"
  }
}

Get text with a longer timeout:

{
  "command": "get_text",
  "params": {
    "selector": "css=.dynamic-content",
    "output_variable": "content",
    "timeout": 15000
  }
}

Behavior Notes

If the element is not found within the timeout period, the command fails and the automation stops (unless wrapped in a try_catch).
The text content includes all visible text within the element, including text in child elements.
If the element contains no text, an empty string is stored.
The result is stored as a string, even if the text looks like a number.

get_attribute

The get_attribute command gets the value of an HTML attribute from an element and stores it in a runtime variable.

Description

HTML attributes contain metadata about elements. Common attributes include:

href — the URL of a link
src — the source URL of an image or script
data-id — custom data attributes
title — tooltip text
alt — alternative text for images

Use get_attribute to capture these values for use in later commands.

Parameters

Parameter	Type	Required	Default	Description
`selector`	string	✅ Yes	`""`	The CSS, XPath, or text selector for the element.
`attribute`	string	✅ Yes	`""`	The name of the HTML attribute to read (e.g., `href`, `src`, `data-id`).
`output_variable`	string	No	`""`	The name of the variable to store the attribute value in.
`iframe`	string	No	`""`	If the element is inside an iframe, provide the iframe’s selector here.
`timeout`	number	No	`5000`	Maximum time in milliseconds to wait for the element to appear.

Examples

Get a link’s URL:

{
  "command": "get_attribute",
  "params": {
    "selector": "css=a.product-link",
    "attribute": "href",
    "output_variable": "product_url"
  }
}

Get an image source:

{
  "command": "get_attribute",
  "params": {
    "selector": "css=img.product-image",
    "attribute": "src",
    "output_variable": "image_url"
  }
}

Get a custom data attribute:

{
  "command": "get_attribute",
  "params": {
    "selector": "css=.product-card",
    "attribute": "data-product-id",
    "output_variable": "product_id"
  }
}

Behavior Notes

If the element is found but the attribute doesn’t exist, null is stored in the variable.
If the element is not found within the timeout, the command fails.
Attribute values are always returned as strings.
For boolean attributes (like checked or disabled), the attribute value is an empty string if present, or null if absent. Use get_property instead for boolean values.

get_property

The get_property command gets a JavaScript DOM property from an element and stores it in a runtime variable.

Description

DOM properties are different from HTML attributes. While attributes are defined in the HTML markup, properties are the live JavaScript values of DOM elements. Common DOM properties include:

value — the current value of an input field
checked — whether a checkbox is checked (boolean)
innerText — the visible text content
innerHTML — the HTML content including tags
scrollHeight — the total scrollable height of an element

Use get_property when you need the live JavaScript value rather than the static HTML attribute.

Parameters

Parameter	Type	Required	Default	Description
`selector`	string	✅ Yes	`""`	The CSS, XPath, or text selector for the element.
`property`	string	✅ Yes	`""`	The name of the DOM property to read (e.g., `value`, `checked`, `innerText`).
`output_variable`	string	No	`""`	The name of the variable to store the property value in.
`iframe`	string	No	`""`	If the element is inside an iframe, provide the iframe’s selector here.
`timeout`	number	No	`5000`	Maximum time in milliseconds to wait for the element to appear.

Examples

Get an input field’s current value:

{
  "command": "get_property",
  "params": {
    "selector": "css=input#search-box",
    "property": "value",
    "output_variable": "search_value"
  }
}

Check if a checkbox is checked:

{
  "command": "get_property",
  "params": {
    "selector": "css=input#newsletter-opt-in",
    "property": "checked",
    "output_variable": "is_opted_in"
  }
}

Get the scroll height of a container:

{
  "command": "get_property",
  "params": {
    "selector": "css=.scroll-container",
    "property": "scrollHeight",
    "output_variable": "container_height"
  }
}

Behavior Notes

DOM properties return their native JavaScript types: value returns a string, checked returns a boolean, scrollHeight returns a number.
If the element is not found within the timeout, the command fails.
If the property doesn’t exist on the element, undefined is stored.
For input fields, value returns the current value (what the user typed), not the initial HTML value.

is_visible

The is_visible command checks whether an element is currently visible on the page and stores the result (true or false) in a runtime variable.

Description

Use is_visible to check whether an element exists and is visible before interacting with it. This is useful for:

Checking if a “Load More” button exists before clicking it
Verifying that a success message appeared
Checking if a popup is open
Conditional logic based on element visibility

Parameters

Parameter	Type	Required	Default	Description
`selector`	string	✅ Yes	`""`	The CSS, XPath, or text selector for the element.
`output_variable`	string	No	`""`	The name of the variable to store the result (`true` or `false`).
`iframe`	string	No	`""`	If the element is inside an iframe, provide the iframe’s selector here.
`timeout`	number	No	`5000`	Maximum time in milliseconds to wait for the element to appear.

Examples

Check if a “Load More” button exists:

{
  "command": "is_visible",
  "params": {
    "selector": "css=button.load-more",
    "output_variable": "has_more"
  }
}

Check for a success message:

{
  "command": "is_visible",
  "params": {
    "selector": "css=.success-message",
    "output_variable": "is_success",
    "timeout": 10000
  }
}

Use with conditional logic:

{
  "command": "is_visible",
  "params": {
    "selector": "css=.cookie-banner",
    "output_variable": "show_cookie_banner"
  }
},
{
  "command": "if",
  "params": {
    "condition": {
      "type": "script",
      "expression": "variables.show_cookie_banner === true"
    },
    "then": [
      {
        "command": "click",
        "params": { "selector": "css=.cookie-accept" }
      }
    ]
  }
}

Behavior Notes

is_visible returns true if the element exists in the DOM AND is visible (not hidden by CSS display: none, visibility: hidden, or opacity: 0).
If the element doesn’t exist in the DOM at all, is_visible returns false (it does NOT throw an error).
This makes is_visible safe to use without wrapping in try_catch.
The timeout parameter controls how long to wait for the element to appear before returning false.

Extraction Workflow Example

Here’s a complete example showing how extraction commands work together in a real scraping workflow:

{
  "command": "goto",
  "params": { "url": "https://example.com/products" }
},
{
  "command": "extract_data",
  "params": {
    "dataset": "product_list",
    "output_variable": "products"
  }
},
{
  "command": "for_each",
  "params": {
    "source_type": "variable",
    "variable": "products",
    "as": "product",
    "do": [
      {
        "command": "goto",
        "params": { "url": "${product.url}" }
      },
      {
        "command": "get_text",
        "params": {
          "selector": "css=h1.product-title",
          "output_variable": "detail_title"
        }
      },
      {
        "command": "get_attribute",
        "params": {
          "selector": "css=img.main-image",
          "attribute": "src",
          "output_variable": "image_url"
        }
      },
      {
        "command": "extract_data",
        "params": {
          "dataset": "product_details",
          "parent_data_from": "product"
        }
      }
    ]
  }
}

This workflow:

Navigates to a product listing page.
Extracts the product list into a variable.
Loops through each product.
Visits each product’s detail page.
Extracts additional details.
Merges the details with the parent product data.