For years now, our Bot has offered the ability to blacklist certain words and phrases, to allow you to have a simple auto-moderation experience. Today, we’re taking this auto-moderation to the next level, using AI.
The new AI Moderation settings are now rolled out to all chatrooms & organizations. Everybody on the Business tier is able to use AI Moderation right away, whereas other tiers are able to unlock it for an additional $25USD/month.
AI Moderation looks at the context of the message and can intelligently analyze whether or not it contains hateful content. For instance, if a user says “It sucks that it’s raining,” that’s perfectly fine — but if a user says “[user] sucks,” the AI will know that the user is being a nuisance. Same with other forms of hateful content; saying the names of colors will be permissible, but engaging in racist attacks will be blocked.
You can take it to the next level by having the Bot analyze for profanity, as well. AI will see through clever attempts to misspell and sneak by filters, and take action accordingly. If your chatroom is full of adults, though, you can keep the AI simple and only target hateful content, allowing profanity to be used in the chat, as long as it’s not done in a harmful way.
To get started with AI Moderation, navigate to your chatroom settings and scroll down to the “AI Moderation” button. You can also read our documentation for more details about how AI can help moderate your chatroom.
Enjoy!
—Minnit