NSFW content has been a source of complaints and "show less like this" interactions on 💖For You.

Let's look at the data. I started logging the labels applied to the posts shown in For You and here are some stats for content with no labels vs with labels from this morning:

flag                   impressions    like_%   show_less_%   show_more_%
--------------------  ------------  --------  ------------  ------------
clean                     11702220      1.30        0.0144        0.0031

porn                       1295504      2.01        0.0313        0.0124
sexual                      509315      1.64        0.0210        0.0100
nudity                      110215      1.73        0.0354        0.0145
sexual_figurative            29592      1.69        0.0169        0.0169
graphic_media                19093      1.60        0.0576        0.0105

Indeed, the content with the labels is ~2x more likely to get "show less like this" (0.03% vs 0.014%). Interestingly, "graphic media" gets the highest rate of "show less". However, the sample size for this label is too small to draw strong conclusion.

What is more interesting is that labeled content has a significantly higher like rate (1.3% -> 2%) and a 4x higher rate of "show more like this" (0.0031% -> ~0.012%). The "show more" increase could be due to the private nature of this interaction - nobody else but the feed operator gets to know about this and users could be more comfortable using this vs publicly liking.

I long wanted to find out how to make the proper use of the content labels. When I created https://foryou.club/also-liked I saw that when you enter a sfw post as the seed item you would often see nsfw content in the recommendations just because people who happened to like it also liked nsfw posts.

In such a simple "also liked" case I decided to filter out content that is more nsfw than the seed item. If the user has entered a sfw seed post then they have not positively expressed that they want nsfw content. Before we show nsfw content we probably want some explicit signal from the user that they are seeking such content.

Would such logic work in For You more generally? Often people complain that they get nsfw content when they haven't liked such content at all. The simplest thing we could do is find the labels of all content the user has liked in the past and not show content that is "more nsfw" than their past likes. For example, if the user liked at most "sexual figurative" then we would keep recommending such content but we would not expose them to "porn".

The labels though don't capture the full spectrum of nsfw flavors. A user may like one type of "sexual" content but not all. Could we capture this?

Here is an idea - show a nsfw post only if it was liked by at least one co-liker with whom you share a like on a post with the same nsfw label.

Suppose we are generating For You for Alice:

  • If Alice liked sfw posts that Bob liked then when Bob likes nsfw posts we ignore them because Alice has not shown that she shares Bob's nsfw tastes.

  • If Alice likes nsfw posts that Carol liked then other nsfw posts that Carol liked get through.

I added this logic to For You and you can test it out in the playground by enabling this option:

A/B test

As always, to see if this change is any good I have started an A/B test where 50% of the users will see the old logic and 50% will have the new logic. I expect that in the "new logic" group For You will show less nsfw content to people who don't want it and it would be reflected in the metrics:

  • we would see a lower rate of "show less" for nsfw content

  • the volume of nsfw labeled post impressions is not affected too much

  • overall like rates per feed load and per user stay the same

  • the number of feed loads per user is slightly higher because users are not turned away when then come across unwanted content