Transformations

Transformations are processing steps that modify extracted data after it’s been pulled from the page. They let you clean up text, extract specific parts, convert data types, and apply custom logic — all without writing code.

Think of transformations as a filter or processor that sits between the raw extracted data and your final output. You can chain multiple transformations together, and they run in order from top to bottom.

Available Transformations

RTILA X provides 11 transformation types:

Transformation	What It Does	Example
trim	Removes whitespace from start and end	" $29.99 " → "$29.99"
prefix	Adds text to the beginning	"29.99" → "$29.99"
suffix	Adds text to the end	"29.99" → "29.99 USD"
replace	Finds and replaces text	"$29.99" → "29.99"
regex	Extracts using a regular expression	"$29.99 USD" → "29.99"
extract_regex	Extracts all regex matches as an array	"a@b.com, c@d.com" → ["a@b.com", "c@d.com"]
join	Joins an array into a string	["a", "b"] → "a, b"
cast	Converts to integer or float	"29.99" → 29.99
json_parse	Parses a JSON string into an object	'{"a":1}' → {"a": 1}
json_path	Extracts a value from JSON using a path	{"items":[{"name":"X"}]} → "X"
script	Runs custom JavaScript	return value.toUpperCase();

trim

Description

Removes whitespace (spaces, tabs, newlines) from the beginning and end of the extracted value. This is the most commonly used transformation, as web pages often have extra whitespace around text.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "trim"

No additional parameters are needed.

Example

Input value: " $29.99 "

After trim: "$29.99"

Behavior

• Removes spaces, tabs (\t), newlines (\n), and other whitespace characters
• Only removes from the start and end, not from the middle
• If the value is already trimmed, nothing changes
• If the value is null or empty, it remains unchanged

When to Use

Scenario	Use trim?
Text has leading/trailing spaces	✅ Yes
Text has spaces in the middle	❌ No (use replace)
Value is already clean	⚠️ Optional (doesn’t hurt)

Tip: Add trim as the first transformation in almost every text extraction. Web pages frequently have invisible whitespace that you don’t want in your data.

prefix

Description

Adds a specified text string to the beginning of the extracted value. Useful for adding currency symbols, URL prefixes, or other text that should appear before the data.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "prefix"
value	string	✅ Yes	—	The text to add to the beginning

Example

Input value: "29.99"

Transformation:

{
  "type": "prefix",
  "value": "$"
}

After prefix: "$29.99"

Behavior

• The prefix is added only if the value doesn’t already start with it
• If the value already starts with the prefix, it’s not added again
• If the value is null or empty, the prefix is still added

When to Use

Scenario	Example
Adding currency symbol	"29.99" → "$29.99"
Adding URL prefix	"/products/123" → "https://example.com/products/123"
Adding category prefix	"headphones" → "Category: headphones"

Tips

Tip	Why
Check for existing prefix	prefix won’t add duplicates
Use for URL construction	Combine with page_url to build absolute URLs
Order matters	prefix runs before subsequent transformations

suffix

Description

Adds a specified text string to the end of the extracted value. Useful for adding units, currency codes, or other text that should appear after the data.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "suffix"
value	string	✅ Yes	—	The text to add to the end

Example

Input value: "29.99"

Transformation:

{
  "type": "suffix",
  "value": " USD"
}

After suffix: "29.99 USD"

Behavior

• The suffix is added to the end of the value
• If the value is null or empty, the suffix is still added

When to Use

Scenario	Example
Adding currency code	"29.99" → "29.99 USD"
Adding units	"500" → "500 GB"
Adding file extension	"report" → "report.pdf"

replace

Description

Finds text matching a pattern and replaces it with new text. Uses JavaScript regular expressions for powerful pattern matching.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "replace"
find	string	✅ Yes	—	The regex pattern to find
replace_with	string	✅ Yes	—	The text to replace matches with
flags	string	No	"g"	Regex flags (e.g., g for global, i for case-insensitive)

Example

Remove dollar signs:

Input value: "$29.99 USD"

Transformation:

{
  "type": "replace",
  "find": "[$]",
  "replace_with": "",
  "flags": "g"
}

After replace: "29.99 USD"

Remove all whitespace:

Input value: "Wireless Bluetooth Headphones"

Transformation:

{
  "type": "replace",
  "find": "\\s+",
  "replace_with": " ",
  "flags": "g"
}

After replace: "Wireless Bluetooth Headphones"

Regex Flags

Flag	Description	Example
g	Global — replace all occurrences, not just the first	"aaa" → replace all as
i	Case-insensitive — match regardless of case	"Hello" matches "hello"
m	Multiline — ^ and $ match line starts/ends	For multiline text

Common Replace Patterns

Pattern	Replaces	Example
[$]	Dollar signs	"$29.99" → "29.99"
\\s+	Multiple whitespace	"a b" → "a b"
[^0-9.]	Everything except digits and dots	"Price: $29.99" → "29.99"
,	Commas	"1,234" → "1234"
https?://	URL protocol	"https://example.com" → "example.com"

Behavior

• Uses JavaScript regex syntax
• If find pattern matches nothing, the value is unchanged
• The flags parameter defaults to "g" (global replace)
• Special regex characters in find must be escaped with backslash

Tips

Tip	Why
Escape special regex characters	. matches any character. Use \\. for a literal dot
Use g flag for all occurrences	Without g, only the first match is replaced
Test regex patterns first	Use an online regex tester to verify your pattern
Combine with trim	Replace may leave whitespace. Add trim after.

regex

Description

Extracts a specific part of the value using a regular expression. Unlike replace which replaces text, regex extracts a captured group from the value.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "regex"
pattern	string	✅ Yes	—	The regex pattern with capture groups
group	number	No	0	The capture group number to extract (0 = full match)
flags	string	No	""	Regex flags
default_value	any	No	null	Value to return if no match is found

Example

Extract price from text:

Input value: "Price: $29.99 USD"

Transformation:

{
  "type": "regex",
  "pattern": "\\$(\\d+\\.\\d{2})",
  "group": 1
}

After regex: "29.99"

Extract email from text:

Input value: "Contact us at support@example.com for help"

Transformation:

{
  "type": "regex",
  "pattern": "([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,})",
  "group": 1
}

After regex: "support@example.com"

Capture Groups

Group Number	What It Returns	Example
0	The full match	"Price: $29.99" → "Price: $29.99"
1	First capture group (...)	"Price: $29.99" → "$29.99"
2	Second capture group	"John (age 30)" → "age 30"

Behavior

• If the pattern doesn’t match, default_value is returned (or null if not set)
• The group parameter specifies which capture group to extract
• Group 0 is the full match. Group 1 is the first (...) in the pattern.
• If the pattern has no capture groups, use group 0

Tips

Tip	Why
Use capture groups (...)	They let you extract specific parts of the match
Set default_value	Prevents null when pattern doesn’t match
Test patterns first	Use an online regex tester
Use group: 1 for first capture	Most common use case

extract_regex

Description

Extracts all matches of a regular expression as an array. Unlike regex which extracts a single match, extract_regex returns all occurrences.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "extract_regex"
pattern	string	✅ Yes	—	The regex pattern to match
flags	string	No	"g"	Regex flags (default is global)
group	number	No	0	The capture group to extract from each match

Example

Extract all emails from text:

Input value: "Contact: john@example.com or jane@test.org for support"

Transformation:

{
  "type": "extract_regex",
  "pattern": "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}",
  "flags": "g"
}

After extract_regex: ["john@example.com", "jane@test.org"]

Behavior

• Returns an array of all matches
• If no matches are found, returns an empty array []
• Use group to extract a specific capture group from each match
• The flags parameter defaults to "g" (global)

When to Use

Scenario	Use extract_regex?
Extract all emails from text	✅ Yes
Extract all URLs from text	✅ Yes
Extract all phone numbers	✅ Yes
Extract a single value	❌ No (use regex instead)

join

Description

Joins an array into a single string using a separator. This is the reverse of split (available via list_operation command) and is useful for converting arrays into readable strings.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "join"
separator	string	No	", "	The separator to use between array items

Example

Input value: ["red", "blue", "green"]

Transformation:

{
  "type": "join",
  "separator": ", "
}

After join: "red, blue, green"

With different separator:

{
  "type": "join",
  "separator": " | "
}

After join: "red | blue | green"

Behavior

• If the input is not an array, the value is returned unchanged
• If the array is empty, returns an empty string ""
• The separator is placed between each item

When to Use

Scenario	Use join?
Convert array of tags to string	✅ Yes
Combine multiple values into one field	✅ Yes
Create comma-separated lists	✅ Yes
Input is already a string	❌ No (not needed)

cast

Description

Converts the extracted value to a numeric type (integer or float). This is essential when you need to perform calculations or sort numerically.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "cast"
to	string	✅ Yes	—	Target type: "integer" or "float"
default_value	any	No	0	Value to return if conversion fails

Example

Convert price string to float:

Input value: "29.99"

Transformation:

{
  "type": "cast",
  "to": "float",
  "default_value": 0
}

After cast: 29.99 (number)

Convert quantity to integer:

Input value: "5 items"

Transformation:

{
  "type": "cast",
  "to": "integer",
  "default_value": 0
}

After cast: 5 (number)

Cast Types

Target Type	Description	Example Input	Output
integer	Whole number (no decimals)	"29.99"	29
float	Decimal number	"29.99"	29.99

Behavior

• The input string is parsed to extract the numeric value
• Non-numeric characters are stripped before conversion
• If conversion fails, default_value is returned
• integer truncates decimals (29.99 → 29)
• float preserves decimals (29.99 → 29.99)

Tips

Tip	Why
Use float for prices	Preserves decimal values
Use integer for counts	Whole numbers only
Set default_value	Prevents errors when conversion fails
Combine with replace first	Remove currency symbols before casting

json_parse

Description

Parses a JSON string into a JavaScript object or array. This is useful when the extracted value is a JSON string that you need to work with as structured data.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "json_parse"
default_value	any	No	null	Value to return if parsing fails

Example

Input value: '{"name": "Headphones", "price": 29.99}'

Transformation:

{
  "type": "json_parse",
  "default_value": {}
}

After json_parse: {"name": "Headphones", "price": 29.99} (object)

Behavior

• If the input is valid JSON, it’s parsed into an object/array
• If parsing fails, default_value is returned
• If the input is already an object (not a string), it’s returned as-is

When to Use

Scenario	Use json_parse?
Extracted value is a JSON string	✅ Yes
Need to access nested JSON fields	✅ Yes (combine with json_path)
Value is already an object	❌ No (not needed)
Value is plain text	❌ No (will fail)

json_path

Description

Extracts a specific value from a JSON object or array using a dot-notation path. This is used after json_parse to navigate into nested JSON structures.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "json_path"
path	string	✅ Yes	—	Dot-notation path to the value

Path Syntax

Path	Accesses	Example
name	Top-level field	{"name": "X"} → "X"
data.items	Nested field	{"data": {"items": []}} → []
data.items[0]	First array item	{"data": {"items": [{"id": 1}]}} → {"id": 1}
data.items[0].name	Field in first array item	{"data": {"items": [{"name": "X"}]}} → "X"
data.items[2].price	Field in third array item	Accesses third item’s price

Example

Input value (after json_parse):

{
  "data": {
    "products": [
      {"name": "Headphones", "price": 29.99},
      {"name": "Speaker", "price": 49.99}
    ]
  }
}

Transformation:

{
  "type": "json_path",
  "path": "data.products[0].name"
}

After json_path: "Headphones"

Behavior

• If the path doesn’t exist, returns undefined
• Array indices are 0-based (first item is [0])
• Supports deep nesting: a.b.c.d.e
• If the input is not an object, returns undefined

Tips

Tip	Why
Use after json_parse	json_path works on parsed objects
Array indices are 0-based	First item is [0], second is [1]
Chain with json_parse	Parse first, then navigate
Check path exists	Non-existent paths return undefined

script

Description

Runs custom JavaScript code to transform the value. This is the most flexible transformation, allowing you to write any JavaScript logic. The current value is available as the value variable.

Parameters

Parameter	Type	Required	Default	Description
type	string	✅ Yes	—	Set to "script"
code	string	✅ Yes	—	JavaScript code. Must return the transformed value.
default_value	any	No	null	Value to return if the script throws an error

Example

Convert to uppercase:

Input value: "wireless headphones"

Transformation:

{
  "type": "script",
  "code": "return value.toUpperCase();",
  "default_value": ""
}

After script: "WIRELESS HEADPHONES"

Calculate discounted price:

Input value: "29.99"

Transformation:

{
  "type": "script",
  "code": "const price = parseFloat(value); return (price * 0.85).toFixed(2);",
  "default_value": "0"
}

After script: "25.49"

Extract domain from URL:

Input value: "https://www.example.com/products/123"

Transformation:

{
  "type": "script",
  "code": "try { return new URL(value).hostname; } catch(e) { return value; }",
  "default_value": ""
}

After script: "www.example.com"

Script Rules

Rule	Description
Must use return	The script must return the transformed value
value variable available	The current extracted value is in value
Use try/catch	Handle errors to prevent automation failures
Set default_value	Returned if the script throws an error
JavaScript syntax	Standard JavaScript. No Node.js or browser APIs.

Common Script Patterns

Pattern	Code	Result
Uppercase	return value.toUpperCase();	"hello" → "HELLO"
Lowercase	return value.toLowerCase();	"HELLO" → "hello"
Parse float	return parseFloat(value);	"29.99" → 29.99
Round number	return Math.round(parseFloat(value));	"29.7" → 30
Get hostname	return new URL(value).hostname;	URL → domain
Remove HTML tags	return value.replace(/<[^>]*>/g, '');	HTML → plain text
Split and get first	return value.split(',')[0].trim();	"a,b,c" → "a"

Tips

Tip	Why
Always use try/catch	Prevents errors from stopping the automation
Set default_value	Provides a fallback if the script fails
Keep scripts simple	Complex logic is harder to debug
Test scripts separately	Verify your JavaScript works before adding it
Use other transformations first	Prefer trim, replace, cast before script

Chaining Transformations

Transformations are applied in order from top to bottom. The output of one transformation becomes the input of the next. This means order matters.

How Chaining Works

Original Value → Transformation 1 → Transformation 2 → Transformation 3 → Final Output

Each transformation receives the result of the previous transformation.

Example: Cleaning a Price

Original extracted value: " Price: $29.99 USD "

Transformation chain:

Step	Transformation	Input	Output
1	trim	" Price: $29.99 USD "	"Price: $29.99 USD"
2	replace (find: "Price: $", replace: "")	"Price: $29.99 USD"	"29.99 USD"
3	replace (find: " USD", replace: "")	"29.99 USD"	"29.99"
4	cast (to: "float")	"29.99"	29.99

Final output: 29.99 (number)

Order Matters

Consider these two different orders:

Order A: trim → cast

• Input: " 29.99 "
• After trim: "29.99"
• After cast: 29.99 ✅ Works!

Order B: cast → trim

• Input: " 29.99 "
• After cast: 29.99 (whitespace stripped during parsing)
• After trim: 29.99 (trim on a number does nothing)
• Result: 29.99 ✅ Also works, but trim was unnecessary

Order C: cast → replace

• Input: "$29.99"
• After cast: NaN (can’t parse $29.99 as number) ❌ Fails!

Order D: replace → cast

• Input: "$29.99"
• After replace (remove $): "29.99"
• After cast: 29.99 ✅ Works!

Lesson: Always clean the data before converting types. Remove unwanted characters before casting.

Chaining Tips

Tip	Why
Clean before converting	Remove symbols/whitespace before cast
Trim first	Remove whitespace before other operations
Extract before transforming	Use regex to extract, then transform the result
Test each step	Verify each transformation produces the expected output
Don’t over-chain	3-5 transformations is usually enough

Adding Transformations in the Visual Builder

Step-by-Step

1. Open your project in the Visual Builder.
2. Switch to the Data Extraction tab.
3. Select the dataset containing the property you want to transform.
4. Select the specific property in the properties list.
5. Click Edit Transformations in the property configuration panel.
6. The Transformation Editor modal opens.

In the Transformation Editor

7. Choose a transformation type from the dropdown.
8. Click Add.
9. Configure the transformation parameters.
10. Add more transformations as needed.
11. Drag to reorder transformations if needed.
12. Click Save & Close.

Transformation Editor Features

Feature	Description
Add transformation	Select type from dropdown, click Add
Edit parameters	Each transformation shows its specific parameters
Reorder	Drag transformations to change order
Delete	Click the trash icon to remove a transformation
Preview	The live preview table updates after saving

Editing Transformation Order

The order of transformations is critical. In the Transformation Editor:

• Transformations are listed from top to bottom.
• They execute from top to bottom.
• Drag to reorder if needed.
• The first transformation receives the raw extracted value.
• Each subsequent transformation receives the output of the previous one.

Transformation Decision Guide

Use this guide to choose the right transformation for your data:

Your Data Problem	Recommended Transformation	Example
Extra whitespace around text	trim	" $29.99 " → "$29.99"
Need to add text before value	prefix	"29.99" → "$29.99"
Need to add text after value	suffix	"29.99" → "29.99 USD"
Need to remove specific text	replace	"$29.99" → "29.99"
Need to extract a pattern	regex	"Price: $29.99" → "29.99"
Need to extract all matches	extract_regex	"a@b.com c@d.com" → ["a@b.com", "c@d.com"]
Need to join an array	join	["a", "b"] → "a, b"
Need to convert to number	cast	"29.99" → 29.99
Value is a JSON string	json_parse	'{"a":1}' → {"a": 1}
Need to navigate JSON	json_path	{"data":{"x":1}} → 1
Need custom logic	script	Any JavaScript transformation

Common Transformation Recipes

Here are ready-to-use transformation chains for common scenarios:

Recipe: Clean Price

Problem: Price extracted as " $29.99 USD "

Chain:

1. trim — Remove whitespace
2. replace — Find: [$], Replace: "", Flags: "g" — Remove dollar sign
3. replace — Find: " USD", Replace: "" — Remove currency code
4. cast — To: "float" — Convert to number

Result: 29.99

Recipe: Extract Email from Text

Problem: Text contains "Contact: support@example.com for help"

Chain:

1. regex — Pattern: ([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}), Group: 1

Result: "support@example.com"

Recipe: Build Absolute URL

Problem: Link extracted as "/products/123"

Chain:

1. prefix — Value: "https://example.com"

Result: "https://example.com/products/123"

Recipe: Convert to Uppercase and Trim

Problem: Text extracted as " wireless headphones "

Chain:

1. trim — Remove whitespace
2. script — Code: return value.toUpperCase();

Result: "WIRELESS HEADPHONES"

Recipe: Parse JSON and Extract Field

Problem: Value is a JSON string '{"product": {"name": "Headphones", "price": 29.99}}'

Chain:

1. json_parse — Parse the JSON string
2. json_path — Path: "product.name"

Result: "Headphones"