After programmatically looking for scams on TikTok, I decided to try to do the same on YouTube. After a brief look into comments in finance-related channels, I found a very popular pattern of scams that's designed to avoid Google's moderation models.

Initially, a thread of comments is created with seemingly non-malicious questions and replies. The top comment in thread gets a lot of likes, likely from other accounts of attackers. 

After a few replies in the thread, scammers setup the real trap: they mention some financial advisor by name. A few other people quickly join the thread and reply how the advisor helped them as well. 

In some cases, the thread also includes a way to find the advisor ("she is on telegram" or "just google her"). 

None of the comments in the thread include links, which are usually huge red flags for moderation ML models (even if transformed like "http@@s scammer com").  Furthermore, having to find the "financial advisor" on the internet makes the victim trust the scammer more.

It's also interesting how first few comments are never related to the scam itself and do not promote any particular advisor. Instead, these comments act as a good setup ("what can i do in this situation?") to the real scam. Even if the advisor-mentioning comment is deleted later, scammers always can add another comment to the previously created thread. 

Dataset & Approach

In order to find more examples of these scams, I've chosen two relatively small YouTube channels related to retirement planning and investing: Tom Crosshill and Retire Confidently. I downloaded comments with replies from 230 videos between two channels using RapidAPI.

In order to filter scams, I didn't create a tool to catch this specific pattern of scams. Instead, I used just slightly adapted prompt from the previous article. 

async def analyze_comment(comment_text, video_id):
    """Analyze a single YouTube comment for scam content using OpenAI"""
    prompt = f"""Analyze this YouTube comment for potential scam content. Look for:
- Investment scams (fake crypto, trading schemes, get-rich-quick)
- Romance/relationship scams 
- Phishing attempts
- Fake business opportunities
- Pyramid schemes or MLMs
- Identity theft attempts
- Fake giveaways or contests
- Suspicious links or contact requests
- Fake WhatsApp/Telegram contact sharing
- Impersonation of the channel owner or celebrities

Comment: "{comment_text}"
Video ID: {video_id}

Classify as either "scam" or "not_scam" and provide a brief explanation."""


response = await client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are an expert at detecting scams in YouTube comments. Respond with a JSON object containing 'classification' (either 'scam' or 'not_scam') and 'explanation' fields."},
        {"role": "user", "content": prompt}
    ],
    response_format={"type": "json_object"},
    temperature=0.1
)

result = json.loads(response.choices[0].message.content)

Results

This script classified 640 out of 5149 comments (11%) as scams. That's a lot for two small YouTube channels!

Example comments that were caught:

False positives

Both channel authors had comments classified as scams.

According to LLM: "The comment promotes a specific website for investment training, which raises red flags for potential investment scams. The inclusion of a link to an external site, especially one that claims to offer easy ways to optimize portfolios, is a common tactic used in scams. Additionally, the comment attempts to discredit other recommendations, which is a typical strategy to steer users away from legitimate advice and towards the scammer's offering."

This is a relatively easy fix — the channel author can be special cased. 

A comment about using social security in a country with lower cost of living was mistakenly classified as a scam: "SS provides 100% of my needs with so much left over I paid off 20k in CC debt in last few years, resulting in even more disposable income. I rent a furnished 1 bdrm, 2 bath seaview apt  /w pool and maid service for $500 including utilities. Thailand." According to LLM: "The comment promotes an unrealistic financial situation and implies a get-rich-quick scheme...".

Another comment that was mistakenly classified as scam: "I have my own indicator "common sense"! When I have doubled my money or seen stocks increase by $60-100 shar; well that's my indicator. Foolish men stay until the end!".  According to LLM: "The comment suggests a get-rich-quick mentality by claiming to have a personal indicator for investment success. Phrases like 'doubled my money' and 'stocks increase by $60-100' imply unrealistic returns...".

Without context of Roth vs traditional IRA discussion, this comment was mistakenly classified as a scam: "Put money into it now and pay taxes now. Most likely, it will never be missed. When eligible, withdraw untaxed AND - HERE'S THE BIGGIE - all the interest you've earned will not be taxed. That's huge. Once you get to retirement age you will be glad you went through your younger years paying taxes and not have to pay now.".

Sometimes, the script would catch self-promotions, but not necessarily scams. Here is one from a real estate advisor in Florida with real YouTube videos on the account: "If LEAVING tax free wealth is your goal, and you live in Florida … I can help you do that."

In general, it seems like the promising next step to deal with false positives is including more context from the source video, the comment thread and author's channel.

Overall, out of 30 comments that I manually looked at, 11 turned out to be false positives. 36% is a huge FP rate, making this initial script unsuitable for production.

False negatives

In order to get examples of false negatives, I looked on "not-scam" comments of users, who had other "scam" comments.

As expected, many "setup" comments (but not all!) within the scam thread were classified as non-malicious. "this is huge! would you mind revealing info of your advisor here please? in dire need of portfolio rebalancing" — classified as "non-scam" with explanation "The comment expresses a genuine interest in financial advice and portfolio management without any overt signs of scam tactics such as suspicious links, requests for personal information, or offers of fake investment opportunities. It appears to be a request for legitimate financial guidance rather than a scam."

The false negative comment in the whole thread.

Interestingly enough, the same account was used for a comment that directly mentioned the financial advisor under another video.

A potential strategy to fix these false negatives would be to include more context: the whole thread and/or other comments from the same user. It also might be needed to instruct LLM directly about this pattern of scams in the prompt.

Reporting to YouTube

I reported 20 comments to YouTube as a test batch to see if these actually going to be deleted. If it works, I'm going to report bigger batches until all 640 comments are covered.

No comments were blocked after a few hours, but I'm still full of hope!