Suppose that I run a website which offers tickets to cultural events and
I already have some collected data from the past - namely views of the
events and purchases of the tickets by the users. For simplicity, I have
the interactions in two .json files with the same structure.
We will use SDK for
communication with Recombee, as it greatly simplifies the process of the
integration. I need the Recombee client, so we will import it from the
already installed package. We will also import all the classes for the
requests in order to spare some typing. Then, I create the instance of
the client. It is initialized with the ID of my database and the secret
token, which I both gained when I created the instant
account, so we will just copy and
paste them from the UI at
admin.recombee.com.
using Recombee.ApiClient;
using Recombee.ApiClient.ApiRequests;
using Recombee.ApiClient.Bindings;
var client = new RecombeeClient("events-example", "PbBaEVxx8ZOj0x3BhGtqfHyi8qQ8rm8rE1JBnSPoCnHwetzO3gjHer96YVAIa14G");
Copy
var recombee = require('recombee-api-client');
var rqs = recombee.requests;
var client = new recombee.ApiClient('events-example', 'PbBaEVxx8ZOj0x3BhGtqfHyi8qQ8rm8rE1JBnSPoCnHwetzO3gjHer96YVAIa14G');
Now, I want to upload some interactions to the recommender system. We
will start by reading and the purchases from the json file and creating
an AddPurchase
request object for each purchase in the file. It takes two mandatory
parameters - the userId and the itemId. We will set the optional
parameter timestamp to time, because the default value is the
current time, but I want a particular time from past. And we will also
set cascadeCreate to true in order to create in the system the yet
non existing items and users.
var client = new RecombeeClient("events-example", "PbBaEVxx8ZOj0x3BhGtqfHyi8qQ8rm8rE1JBnSPoCnHwetzO3gjHer96YVAIa14G");
string[] allLines = File.ReadAllLines(@"purchases.csv");
var parsed = from line in allLines
let row = line.Split(',')
selectnew
{
UserId = row[0],
ItemId = row[1],
Timestamp = DateTime.Parse(row[2], null, System.Globalization.DateTimeStyles.RoundtripKind)
};
var purchases = parsed.Select(x => new AddPurchase(x.UserId, x.ItemId, timestamp: x.Timestamp, cascadeCreate: true));
Copy
var interactions = require('./purchases.json');
var requests = interactions.map((interaction) => {
var userId = interaction['user_id'];
var itemId = interaction['item_id'];
var time = interaction['timestamp'];
returnnew rqs.AddPurchase(userId, itemId, {timestamp: time, cascadeCreate: true});
});
I could send the purchases one by one, using the send method of the
client.
Sending individual requests is very beneficial when you have the
recommender already deployed in production as the system can
immediately modify its recommendations using the just received
interaction.
But uploading larger data from the past with individual requests would
be quite slow, so for sending the list of purchases we will
use the Batch request,
which can encapsulate many requests into a single request.
Now let's run the script and see the result in the Admin UI.
You should see the uploaded items in the Catalog listing. You can also
check interactions of an item by clicking its id.
Now I'll change the code to send the detail views by changing the name
of file with interactions (purchases.csv to detail_views.csv ) and
the name of the class from AddPurchase to
AddDetailView.
var detailViews = parsed.Select(x => new AddDetailView(x.UserId, x.ItemId, timestamp: x.Timestamp, cascadeCreate: true));
Copy
returnnew rqs.AddDetailView(userId, itemId, {timestamp: time, cascadeCreate: true});
After running the changed code, detail views should appear in the
interface.
You can also see the visualization in the KPI console, which is
updated up to every few minutes.
Recommending items to user
Now it's time to get some recommendations based on the uploaded
interactions.
Suppose that user-27 just came to my website. I want to immediately
show him 5 events that he will most likely favor. I'll use the
RecommendItemsToUser for this task.
Now I can show these recommended events to user-27 at my homepage.
Recommending items to item
Let's say that the user likes the recommended events, and clicked one of
them, namely event-32, to see the details. The page of the event
contains box with related events, which are obtained by requesting the
RecommendItemsToItem. The id of the user is passed in the request as well, to
make the related items personalized for user-27.
$recommended = $client->send(newReqs\RecommendItemsToItem('event-32', 'user-27', 5));
echo'Related items to "event-32" for "user-27": ' . print_r($recommended, true) . "\n";
Copy
Console.WriteLine("Items related to \"event-32\" for \"user-27\":");
recommended = client.Send(new RecommendItemsToItem("event-32", "user-27", 5));
foreach(Recommendation rec in recommended.Recomms) Console.WriteLine(rec.Id);
Other two supported recommendation endpoints,
RecommendUsersToItem and
RecommendUsersToUser return users instead of items. For example if I want to know
which users would be most likely interested in attending
event-42, I'll call the
RecommendUsersToItem.
$recommended = $client->send(newReqs\RecommendUsersToItem('event-42', 5));
echo'Users who should be interested in "event-42": ' . print_r($recommended, true) . "\n";
Copy
Console.WriteLine("Users who should be interested in to \"event-42\":");
recommended = client.Send(new RecommendUsersToItem("event-42", 5));
foreach(Recommendation rec in recommended.Recomms) Console.WriteLine(rec.Id);
As you can see sending interactions and getting recommendations is very easy, and took just few lines of code. In the next tutorial I'll show you how to send properties of the items to Recombee and how to use these properties in filtering and boosting according to your business rules.
Items in the Recombee system can have many properties such as title,
description, categories, price and many other. There are two
possibilities how to get your items catalog into Recombee - one is
setting a catalog feed
and the other is uploading the data from your server. This tutorial
covers the second option.
Let's continue with the sample company that sell tickets to cultural
events from previous part of the tutorial.
The events have following properties:
id
name
String
city
String
venue
String
genres
Set
date
Timestamp
price
Double
nyphilharmonic181210
New York Philharmonic: Rachmaninoff
New York
Lincoln Center
["classical music"]
2018-12-10 19:00
94.0
beachboys190321
Beach Boys
New Jersey
Arts Center
["rock music", "rock’n’roll music"]
2019-03-21 20:00
69.0
snoopdogg191102
Snoop Dogg
New York
Terminal 5
["hip-hop", "rap"]
2019-02-11 19:00
42.0
I have these data in a json object, where the keys are IDs of the items
and values contain an object with item properties.
Now, let's see how we can send these data to Recombee, and how they are
used in recommendations. First, I need to define the properties. They
can be thought as columns in relational database. In my case the
properties are title, city, venue, date, genres and price.
client.send(
new rqs.Batch([
new rqs.AddItemProperty('title', 'string'),
new rqs.AddItemProperty('city', 'string'),
new rqs.AddItemProperty('venue', 'string'),
new rqs.AddItemProperty('date', 'timestamp'),
new rqs.AddItemProperty('genres', 'set'),
new rqs.AddItemProperty('price', 'double')
])
);
The properties are now in the place. If there were already some items in
the database, they would have all the values set to NULL.
Setting item values
I start by reading the file, parsing the json and iterating over the
item IDs and corresponding property values. I will send them to Recombee
with SetItemValues requests.
SetItemValues takes the itemId, the property values of the items, and the
optional parameter cascadeCreate which I set to true in order to create items
in the system if they don't exist yet. I use the Batch to speed up the uploading.
requests = []
withopen('items.json') as f:
data = json.loads(f.read())
for item_id,values in data.items():
r = SetItemValues(item_id,
values,
cascade_create=True)
requests.append(r)
res = client.send(Batch(requests))
Copy
ArrayList<Request> requests = newArrayList<>();
try {
// HashMap<itemId, HashMap<name of property, value of property>>
HashMap<String, HashMap<String, Object>> items =
newObjectMapper().readValue(newFileReader("items.json"), HashMap.class);
for (Map.Entry<String, HashMap<String, Object>> entry : items.entrySet()) {
StringitemId= entry.getKey();
HashMap<String, Object> values = entry.getValue();
requests.add(newSetItemValues(itemId, values).setCascadeCreate(true));
}
} catch (IOException e) {
e.printStackTrace();
}
BatchResponse[] res = client.send(newBatch(requests));
Copy
file = File.read('items.json')
requests = JSON.parse(file).map do |item_id, values|
SetItemValues.new(item_id, values, :cascade_create => true)
end
br = Batch.new(requests)
puts client.send(br)
using (StreamReader r = new StreamReader("items.json"))
{
string json = r.ReadToEnd();
Dictionary<string, Dictionary<string, Object>> items =
JsonConvert.DeserializeObject<Dictionary<string, Dictionary<string, Object>>>(json);
var requests = new List<SetItemValues>();
foreach(KeyValuePair<string, Dictionary<string, Object>> entry in items)
{
var itemId = entry.Key;
var values = entry.Value; // Dictionary with names of properties as keys// and values of the properties as values
requests.Add(new SetItemValues(itemId, values, cascadeCreate: true));
}
client.Send(new Batch(requests));
}
Copy
var catalog = require('./items.json');
var requests = [];
for (var itemId in catalog) {
var values = catalog[itemId];
requests.push(new rqs.SetItemValues(itemId, values, {cascadeCreate: true}));
}
client.send(new rqs.Batch(requests));
I can check the uploaded items in the Admin UI.
Other properties that are usually very useful, but were not included in
this example are text descriptions, which are processed by text
mining algorithms, and links to images (property types image and
imageList) that can be used for finding visually similar items using
artificial neural networks and other images processing models.
ReQL filtering allows you to put some restrictions on the recommended items.
Tip
You can use ReQL also for specifying the Business Rules in the Admin
UI
Let's assume I have the catalog of cultural events from previous part
of the tutorial uploaded in my database. It contains following item
properties: title, city, venue, genres, date and price.
I want to get some related items to item
nyphilharmonic181210, which is concert by the New York
Philharmonic. These recommendations will be shown to
user-27. I use RecommendItemsToItem for this purpose:
client.send(
new rqs.RecommendItemsToItem('nyphilharmonic181210', 'user-27', 3, {returnProperties: true})),
callback);
Now I get also the property values of recommended items. As you can see,
the recommended items look pretty related to the concert of the
philharmonic.
JSON
{"recomms":[{"values":{"city":"New York","price":56.0,"venue":"Lincoln Center","title":"Ravel's Daphnis et Chloe and Dvorak","genres":["classical music"],"date":1523318400.0},"id":"ravelsdaphnis180410"},{"values":{"city":"Los Angeles","price":62.0,"venue":"Walt Disney Concert Hall","title":"Los Angeles Philharmonic: Rachmaninoff","genres":["classical music"],"date":1558742400.0},"id":"laphilharmonic190525"},{"values":{"city":"San Francisco","price":58.0,"venue":"Davies Symphony Hall","title":"Czech Philharmonic: Dvorak","genres":["classical music"],"date":1573430400.0},"id":"czechphilharmonic191111"}],"recommId":"295b2aa5-8375-48bc-82dd-e1555ee0b5ec"}
But if the user is interested in the New York Philharmonic concert
which takes place in New York City, there is only slight chance that
the user will attend concert of Los Angeles Philharmonic in LA.
So I want to put restriction on the recommendations to return only
events that take place in New York. This can be easily done by setting
filter 'city' == "New York".
As you can see I use single quotes for accessing values of the
properties and double quotes, which I had to escape, for creating the
New York string.
Now all the events I received take place in New York:
JSON
{"recomms":[{"values":{"city":"New York","genres":["classical music"],"venue":"Lincoln Center","price":56.0,"date":1523318400.0,"title":"Ravel's Daphnis et Chloe and Dvorak"},"id":"ravelsdaphnis180410"},{"values":{"city":"New York","genres":["classical music"],"venue":"Carnegie Hall","price":72.0,"date":1558656000.0,"title":"Staatskapelle Dresden in NY: Schubert"},"id":"staatskapelle190524"},{"values":{"city":"New York","genres":["classical music"],"venue":"Brooklyn Museum","price":63.0,"date":1552780800.0,"title":"Brooklyn Symphony Orchestra: Borodin, Shostakovich"},"id":"brooklynsymphony190317"}],"recommId":"8ef292ca-7f4c-4be0-8f63-fdb9f4a9fee0"}
The problem is that some events may have already passed, and therefore
should not be recommended. To filter these events out I compare the date
property with current time, which obtain by calling ReQL function
now().
The filter therefore becomes 'city' == "New York" and 'date' >= now():
Now all the events are upcoming. Another restriction can be restriction
on genres - let's suppose the user chose to view only items that have
ballet as one of their genres. It is done by appending
"ballet" in 'genres' to the filter:
recommended = client.send(RecommendItemsToItem('nyphilharmonic181210', 'user-27', 3,
return_properties=True,
filter="'city' == \"New York\" AND 'date' >= now() AND \"ballet\" in 'genres'"))
Copy
client.send(
new recombee.RecommendItemsToItem('nyphilharmonic181210', 'user-27', 3, {
returnProperties: true,
filter: "'city' == \"New York\" AND 'date' >= now() AND \"ballet\" in 'genres'"
}),
callback);
Copy
recommended = client.send(newRecommendItemsToItem("nyphilharmonic181210", "user-27", 3)
.setReturnProperties(true)
.setFilter("'city' == \"New York\" AND 'date' >= now() AND \"ballet\" in 'genres'"));
Copy
recommended = client.send(RecommendItemsToItem.new('nyphilharmonic181210', 'user-27', 3,
:return_properties => true,
:filter => "'city' == \"New York\" AND 'date' >= now() AND \"ballet\" in 'genres'"))
Copy
$recommended = $client->send(newReqs\RecommendItemsToItem('nyphilharmonic181210', 'user-27', 3,
['returnProperties' => true,
'filter' => "'city' == \"New York\" AND 'date' >= now() AND \"ballet\" in 'genres'"]));
Copy
recommended = client.Send(new RecommendItemsToItem("nyphilharmonic181210", "user-27", 3,
returnProperties: true,
filter: "'city' == \"New York\" AND 'date' >= now() AND \"ballet\" in 'genres'"));
Copy
client.send(
new rqs.RecommendItemsToItem('nyphilharmonic181210', 'user-27', 3, {
returnProperties: true,
filter: "'city' == \"New York\" AND 'date' >= now() AND \"ballet\" in 'genres'"
}),
callback);
JSON
{"recomms":[{"values":{"city":"New York","price":82.0,"venue":"Metropolitan Opera House","title":"American Ballet Theatre: Harlequinade","date":1564185600.0,"genres":["ballet"]},"id":"americanballet190727"},{"values":{"city":"New York","price":77.0,"venue":"Joyce Theater","title":"BalletX & Raphael Xavier","date":1555027200.0,"genres":["contemporary dance","ballet"]},"id":"balletx190412"},{"values":{"city":"New York","price":92.0,"venue":"David H. Koch Theater","title":"NYCB: Orpheus","date":1561334400.0,"genres":["ballet"]},"id":"nycb190624"}],"recommId":"f05b4d84-fec0-4ef8-8875-eac9bd378c6c"}
Sometimes the filter may depend on the currently viewed item. For
example, we may want to change our filter not to permit only items from
New York, but recommend events from the same city where the event that is
currently viewed by the user takes place.
The context_item function, which retrieves properties of current item, makes achieving
this behavior easy. We will permit only items that have the same city
as context item: 'city' == context_item["city"]. Therefore if we get
recommendations related to an event that takes place in Los Angeles,
we get only events taking place in LA.
recommended = client.send(RecommendItemsToItem('laphilharmonic190525', 'user-27', 3,
return_properties=True,
filter="'city' == context_item[\"city\"] AND 'date' >= now() AND \"ballet\" in 'genres'"))
Copy
client.send(
new recombee.RecommendItemsToItem('laphilharmonic190525', 'user-27', 3, {
returnProperties: true,
filter: "'city' == context_item[\"city\"] AND 'date' >= now() AND \"ballet\" in 'genres'"
}),
callback);
Copy
recommended = client.send(newRecommendItemsToItem("laphilharmonic190525", "user-27", 3)
.setReturnProperties(true)
.setFilter("'city' == context_item[\"city\"] AND 'date' >= now() AND \"ballet\" in 'genres'"));
Copy
recommended = client.send(RecommendItemsToItem.new('laphilharmonic190525', 'user-27', 3,
:return_properties => true,
:filter => "'city' == context_item[\"city\"] AND 'date' >= now() AND \"ballet\" in 'genres'"))
Copy
$recommended = $client->send(newReqs\RecommendItemsToItem('laphilharmonic190525', 'user-27', 3,
['returnProperties' => true,
'filter' => "'city' == context_item[\"city\"] AND 'date' >= now() AND \"ballet\" in 'genres'"]));
Copy
recommended = client.Send(new RecommendItemsToItem("laphilharmonic190525", "user-27", 3,
returnProperties: true,
filter: "'city' == context_item[\"city\"] AND 'date' >= now() AND \"ballet\" in 'genres'"));
Copy
client.send(
new rqs.RecommendItemsToItem('laphilharmonic190525', 'user-27', 3, {
returnProperties: true,
filter: "'city' == context_item[\"city\"] AND 'date' >= now() AND \"ballet\" in 'genres'"
}),
callback);
As you can see, the possibilities of filtering are very broad. Besides
filtering, there is also the possibility to
boost some items or
use properties of users.