# Catalog Feeds

> Source: https://docs.recombee.com/catalog_feeds

> For the complete documentation index, see [llms.txt](/llms.txt).

**Table of contents**

* [A Step-by-Step Video Tutorial on Setting Up a Catalog Feed](#a-step-by-step-video-tutorial-on-setting-up-a-catalog-feed)
* [General Notes](#general-notes)
* [Google Merchant Feed](#google-merchant-feed)  
   * [Format](#format)  
   * [Tips & Tricks](#tips-tricks)
* [Atom / RSS](#atom-rss-feed)  
   * [How to Set Up](#how-to-set-up)  
   * [IDs with Special Characters](#feeds-md5-item-id)
* [Custom XML](#custom-xml-feed)  
   * [How to Set Up](#id3)  
   * [Tips & Tricks](#id11)
* [Custom JSON](#custom-json-feed)  
   * [How to Set Up](#id6)
* [Custom CSV](#custom-csv-feed)  
   * [How to Set Up](#id7)
* [Heureka XML Feed](#heureka-xml-feed)  
   * [Format](#id9)  
   * [Tips & Tricks](#id10)
* [Handling of Removed Items](#feeds-availability-property)

# Catalog Feeds

Setting a catalog feed in the [Recombee Admin UI](https://admin.recombee.com) is an alternative to[uploading the items catalog using our API](/getting_started#getting-started-catalog).

We periodically download your feed and update items accordingly. The default update period is 4 hours - if it does not match your specific needs, please contact [support@recombee.com](mailto:support@recombee.com).

* Supported protocols for fetching: `http`, `https`, `ftp` and `sftp`.
* Supported feeds compressions: `.zip` and `.gz`.

We support the following types of feed formats:

* [Google Merchant XML / TSV](#google-merchant-feed)
* [Atom / RSS](#atom-rss-feed)
* [Custom XML](#custom-xml-feed)
* [Custom JSON](#custom-json-feed)
* [Custom CSV](#custom-csv-feed)
* [Heureka XML](#heureka-xml-feed)

## A Step-by-Step Video Tutorial on Setting Up a Catalog Feed

## General Notes

* Property name _itemId_ is reserved
* Items deleted from feeds are not deleted in Recombee catalog. Instead, they can be updated as _not available_. For more details see [Handling of removed items](#feeds-availability-property).

## Google Merchant Feed

![](/img/headings/catalog-feed-google-merchant.png) 

### Format

* Specification: <https://support.google.com/merchants/answer/7052112?hl=en>
* We support both `.xml` and `.tsv` (tab separated text file) variants. Both `.zip` and `.gz` compression is supported.
* In addition, we support `.csv` (comma separated) variant (not officially supported by Google)

### Tips & Tricks

* You can use item property `availability` for[filtering out](/reql_filtering_and_boosting) from the recommendations the items that are not available anymore.  
The filter that should be passed to[recommendation requests](/api#recommendations) is:  
ReQL  
```  
'availability' == "in stock"  
```  
Only items in stock will be recommended. You can achieve this also by setting the predefined Filter [Business Rule](/scenarios#filters) _Items in stock \[Google Merchant\]_ to your [Scenario](/scenarios).
* Ids of items are case sensitive - please check that you are using the same case as is used in the feed when you[send interactions](/getting_started#getting-started-interactions).
* You can use for example following JavaScript snippet to get recommended items along with their title, description, link, image link and price:  
JS  
```  
client.send(new recombee.RecommendItemsToItem(itemId, userId, 3,  
{  
	returnProperties: true,  
	includedProperties: ['title', 'description', 'link', 'image_link', 'price'],  
	filter: "'availability' == \"in stock\"",  
	scenario: 'related_items'  
}),  
(err, resp) => {  
	if(err) {  
	console.log("Could not load recomms: ", err);  
	return;  
	}  
	console.log(resp);  
	// Show recommendations  
);  
```

For more info see the [JavaScript library](https://github.com/recombee/js-api-client).

## Atom / RSS

Atom / RSS feeds are used by the news websites and also many blog pages or podcasts to provide structured information about the offered content.

![](/img/headings/catalog-feed-rss.png) 

### How to Set Up

1. Set the URL of your Atom / RSS feed.
2. The feed gets processed and a list of properties that were recognized in the feed is presented in the Suggested properties section. You can pick from this list the properties that you want to synchronize to Recombee.  
You can also create a custom new property and set an [XPath](https://www.w3schools.com/xml/xpath_syntax.asp) for it (relative to the item entry). This is needed especially in the case of large _.zip_ feeds which cannot be fully processed in this step to obtain the suggested properties.
3. Continue to _Preview_ and check that you are getting expected data for all the properties. When you are satisfied, confirm the settings and the feed will get scheduled for processing.

[![Setting RSS feed](/img/feeds/setting_feed.gif)](/img/feeds/setting_feed.gif)

### IDs with Special Characters

The IDs of the items in Recombee may consist of digits, latin letters, underscores, colons, minus signs, at sign, and dot.

If the IDs in your feed do not follow this format (the ID is for example the URL of the item), enable the _MD5 Item IDs_ option in the_Settings_. An [MD5 hash](https://en.wikipedia.org/wiki/MD5) of the original ID value is then used as the ID.

You then need to use the MD5 hash also when you send interactions to Recombee and when you request[Items to Item recommendation](/api#request-recommend-items-to-item).

In JavaScript the MD5 hash of the original ID can be obtained for example like this:

HTML

```
<script src="https://cdnjs.cloudflare.com/ajax/libs/blueimp-md5/2.18.0/js/md5.js"></script>

<script type="text/JavaScript">
	var itemId = md5('<original item ID>');

	// Use the itemId when sending interactions to Recombee / asking for Items to Item recommendation
</script>

```

## Custom XML

An XML that contains structured information about the items.

![](/img/headings/catalog-feed-xml.png) 

### How to Set Up

1. Set the URL of your XML feed.
2. Set _Items Path_ \- the [XPath](https://www.w3schools.com/xml/xpath_syntax.asp) to the root element of the items records
3. Set _Item ID Path_ \- Path (relative to the items path) to the element that contains the itemId. If your IDs contain some special characters, enable the _MD5 Item IDs_ option (see [this section](#feeds-md5-item-id) for more details).
4. The feed gets processed and a list of properties that were recognized in the feed is presented in the Suggested properties section. You can pick from this list the properties that you want to synchronize to Recombee.  
You can also create a custom new property and set an [XPath](https://www.w3schools.com/xml/xpath_syntax.asp) for it (relative to the item entry). This is needed especially in the case of large _.zip_ feeds which cannot be fully processed in this step to obtain the suggested properties.
5. Continue to _Preview_ and check that you are getting expected data for all the properties. When you are satisfied, confirm the settings and the feed will get scheduled for processing.

For information about handling items deletion and updating items availability see [Handling of removed items](#feeds-availability-property).

### Tips & Tricks

* Arrays in XML: if we have set item property `categories` and we want to fill its data, we need to define multiple fields with the same ID inside item entry. It's relative path is `/categories`. It will correctly process and assign 3 values to the property.  
xml  
```  
<item>  
<categories>Drama</categories>  
<categories>Horror</categories>  
<categories>Thriller</categories>  
</item>  
```

## Custom JSON

A JSON that contains structured information about the items.

![](/img/headings/catalog-feed-json.png) 

### How to Set Up

1. Set the URL of your JSON feed.
2. Set _Items Path_ \- a path defined by keys of JSON objects to the array with items records  
![JSON Structure Selector](/img/feeds/json_structure_selector.png)
3. Set _Item ID Path_ \- a path (relative to the item entry) to the element that contains the itemId. If your IDs contain some special characters, enable the _MD5 Item IDs_ option (see[this section](#feeds-md5-item-id) for more details).
4. The feed gets processed and a list of properties that were recognized in the feed is presented in the Suggested properties section. You can pick from the list properties that you want to synchronize to Recombee.  
You can also create a new property and set a path of keys for it (relative to the item entry). This is needed especially in the case of large _.zip_ feeds which cannot be fully processed in this step to obtain the suggested properties.
5. Continue to _Preview_ and check that you are getting expected data for all the properties. When you are satisfied, confirm the settings and the feed will get scheduled for processing.

For information about handling items deletion and updating items availability see[Handling of removed items](#feeds-availability-property).

## Custom CSV

A CSV file where each line contains data for a single item.

![](/img/headings/catalog-feed-csv.png) 

It is expected that:

* The IDs of the items are contained in the first column
* The first row of the CSV file is a header (contains names of the columns)

The file is automatically analyzed to determine the used delimiter and quoting.

### How to Set Up

1. Set the URL of your CSV feed.
2. The feed gets processed and a list of properties (columns) that were recognized in the feed is presented in the Suggested properties section. You can pick from this list the properties that you want to synchronize to Recombee. You need to set the correct data type for each of the properties.  
You can also create a custom new property and set the column name for it. This is needed especially in the case of large _.zip_ feeds which cannot be fully processed in this step to obtain the suggested properties.
3. Continue to _Preview_ and check that you are getting expected data for all the properties. When you are satisfied, confirm the settings and the feed will get scheduled for processing.

For information about handling items deletion and updating items availability see [Handling of removed items](#feeds-availability-property).

## Heureka XML Feed

![](/img/headings/catalog-feed-heureka.png) 

### Format

* Specification: <https://sluzby.heureka.cz/napoveda/xml-feed/>

### Tips & Tricks

* You can use item property `AVAILABLE` for[filtering out](/reql_filtering_and_boosting) from the recommendations the items that are not available anymore.  
The filter that should be passed to[recommendation requests](/api#recommendations) is:  
ReQL  
```  
'AVAILABLE'  
```

## Handling of Removed Items

The Feeds processor can automatically set items that have been removed from the feed as unavailable.

To use this feature, pick the item property that determines availability, and provide the value which denotes the unavailable items:

[![Setting Availability property](/img/feeds/feeds_availability_property.png)](/img/feeds/feeds_availability_property.png)

You can then use this item property in your [Filters](/reql).