Making the internet a better place

August 18, 2022 by Matheus Morett

The Problem

When people use GIPHY, they are looking to express themselves in a fun and entertaining way. Certain search results might seem funny and ironic to some but can trigger others who are struggling with mental health.

With this in mind, identifying search terms that may indicate that a person is struggling helps us to ensure that all users have a positive experience.


The Solution

Identifying high-risk terms

Of course, creating an engine to identify these terms in our search is not an easy task. There are countless acronyms and evolving slang associated with mental health issues. When a user is searching for these terms, it can be a clue that they are suffering.

In order to provide a healthy and supportive experience, we chose to build an integration with Koko, a non-profit company that provides an API capable of identifying such terms and suggesting content to support those who may be suffering from mental health issues.

Integrating with Koko

Setup

Integrating with Koko is pretty simple compared to the effect it can have on someone’s life. To start, we chose to use their Python Package, which you can install by running this command:

pip install koko_keywords

After installing, you will need to export your auth tokens which you can get by contacting Koko.

export KOKO_KEYWORDS_AUTH=username:password

Implementation

Now that you’ve installed everything, you are able to import their library, and check if a word/sentence contains a problematic term.

# Import Koko packageimport koko_keywords
# Use the function Match, to check if it's a problematic term
koko_keywords.match("your term")

# We are also able to pass some options, such as categories, and intensity for better filtering. 
koko_keywords.match("your term", "category=abuse:confidence=high:intensity=high")

You can find infos about it at Koko Documentation.

Performance

One of our concerns was to not slow down search results, so to prevent that, we call Koko’s API in parallel with the regular API call to get search results.

Providing support

Now that we’re integrated with Koko, we are able to identify when high-risk terms are searched on GIPHY and display an interactive banner, connecting users who may be suffering to a safer place where they can have better support.

Koko offers a wealth of resources including peer support, self-guided mini-courses, and crisis triage – their studies have shown that “of users who engage with our interventions, 71% feel more hopeful, 42% feel better about their bodies, and 67% feel less self-hatred.”

Conclusion

Mental health is a serious problem, and GIPHY is always trying to ensure our users come away from our platform having had a positive experience. We are glad that orgs like Koko exist and are actively working to provide accessible assistance for everyone.

— Matheus Morett, Software Engineer, Experience team