Category: Uncategorized

Hello,

We are planning to launch a new service and would like to hear your opinion.
The service is called Site Performance Monitoring. It will allow you to control:

• Available website pages, HTTP / HTTPS response code.
• Page load time.
• Ping -% loss, average delay.
• JavaScript page errors.

If you use monitoring you will immediately receive a notification when your website is down. That will allow to take timely actions.

Get statistics of how fast your web pages load for a time feed. This will let you know the changes in the website loading speed and/or the need to optimize the code for the pages.

Network stats allows you to find connectivity problems. Understand how fast your web server responds.

JS errors on the page affect your website performance and speed of pages loading.

We plan to implement Site Performance Monitoring within 3-4 months.

We will be happy to receive your feedback.
Thank you!

In connection with the Joomla 4 release and the development of the anti-spam plugin for the new version of Joomla, you need to reinstall the anti-spam plugin for Joomla 3.x.
You have to go to the control panel of the CleanTalk plugin and delete it:
https://cleantalk.org/help/update-joomla34

Next, install the new version of the anti-spam plugin by CleanTalk. Please follow this guide: https://cleantalk.org/help/install-joomla34

Let us know if you need any help or have any questions.
Thank you for your patience!

One of the unique features of CleanTalk Anti-Spam is the logging of all requests. Unlike other anti-spam solutions, where forbidden requests just disappear and you don’t even know about them, you have the opportunity to view and analyze data, both on forbidden and allowed queries. This allows you to save data, even if they were accidentally deleted from the mail or admin site.

What features provide Anti-Spam Log

First, it allows you to see all requests in one place for analysis and informativeness.

Secondly, it allows you to give feedback in the case of a false blocking of a request.

Anti-Spam Log page

You can use this link to go to your anti-spam log.
https://cleantalk.org/my/show_requests

Consider the possibilities in more detail on the points:

1. Date and time of the request
   
   
2. The status of the request indicates whether the request was blocked or approved by the service and the address of the site on which the event occurred.
   
   
3. Additional menu request management includes:
   
   3.1. View the request details. This item will be discussed in more detail below.
   
   
   3.2. Feedback button. Allows you to inform the service that this request was processed incorrectly.
   
   If the request was mistakenly blocked, you can mark it as Not Spam, in this case, the IP and Email address of the sender will be added to your personal whitelist.
   If the request was incorrectly allowed, you can mark it as Spam, in this case, the IP and Email address of the sender will be added to your personal blacklist. How to use Personal Blacklist.
   
   CleanTalk analyzes user feedback to improve the service. The reasons for erroneous requests can be:
   
   – on the website page there are JS errors that may interfere with the normal execution of the code
   – outdated version of the plugin
   – the plugin receives incorrect data from the web form due to problems with integration
   
   In any case, you can contact our technical support for advice.
   
   
4. IP and email address of the sender. Clicking on the links, you can see the data on the spam activity of these addresses according to the blacklist database.
   “Page URL” is the address of the page of the site where the request was placed.
   “Source” – the source from which the user navigated to your site.
   
   
5. This section shows spam activity statistics for the sender’s IP and email address. Without going to other pages, you can see data on the number of spam requests that were detected by CleanTalk on other websites

Now let’s take a closer look at the Details link.

1. The request status is Denied or Allowed. A request is when a visitor submits a completed form on a website, such as Comment/Registration/Contact form.
   
   
2. Sender’s data: date/time, nickname, email and IP address.
   
   
3. Request ID, this is service information.
   
   
4. The URL of the page of the site on which the request was made.
   
   
5. The previous page of the site from which the user went to the page on which the request was made (Comment / Registration / Contact form).
   
   
6. The source from which the user came to your site, if it was a Google search, the source will be www.google.com If this is a direct entry to the site, it will be unknown – since in this case, it is impossible to determine exactly. Only for WordPress Plugin.
   
   
7. URL grouping, this option only works for WordPress and with the Store visited URLs option enabled. If you enable this option in the CleanTalk Anti-Spam settings, the plugin will collect data on the last 10 pages that the user visited before sending the request. Pages are grouped by opening time. And you can always know which pages/information motivated the user to subscribe / register or other actions.
   
   
8. The text of the comment/contact form.
   
   
9. The result of the anti-spam check and the reason for blocking. In this case:
   
   – Sender’s IP or email are blacklisted and have high spam activity (*@cl*******.org is a test email for testing)
   – Private list deny is blocking by personal blacklists, in this case it was blocking by blacklist Stop Words, the word “test” is in the blacklist.
   
   
10. Feedback button, you can mark the request as Not Spam or Spam.

One of the features of CleanTalk Anti-Spam is the processing of a user’s request even if he filled out the form incorrectly, for example, he made a mistake in the email address. In this case, the form will give an error message that the form is filled in incorrectly. Some users do not pay attention to this and believe that the information they sent and leave the site, while the information is lost, because it does not fall into the backend of the site. CleanTalk allows you to view such a request and message text in the dashboard. CleanTalk transfers the completed form fields, which can be viewed in the Info field.

Look at the screenshot

1. Since this is a comment form and there is no email address in the request, this means that this form was filled out incorrectly and the email address was specified with an error.
   
   
2. Antispam plugin for WordPress, Drupal 7, Drupal 8, Bitrix is able to intercept errors that the form gives and these errors can be viewed in the request. Notice: this functionality is not supported for all forms since there is not always a technical possibility for this.
   
   
3. The text that was sent by the user.

What to do if you do not want to transmit or store information.

1. You can prohibit the storage of approved requests. Learn more.
   The approved requests will be scanned for spam, but no information about them will be saved.
2. You can add site pages to exclusions for anti-spam checks. In this case, antispam protection will not work on these pages.
3. You can add form fields to exceptions, these fields will not be processed by the antispam service.
4. You can delete information at any time on any request in the dashboard of the site.

If you have any questions, you can ask them in the comments or create a personal ticket, we will always be happy to help you.

We have received several requests about protecting website pages of search results from spambots.

At a glance the solution is quite simple — remove the search results page in “robots.txt”, example:

User-agent: *
Disallow: /search

But further analysis showed that it won’t be a 100% solution and there are many more problems which couldn’t be fixed by just directive “Disallow” and which are being ignored even by big corporations.

Anyone who is aware of crawling budget knows that it brings problems about SEO.

One of the Google Webmaster Guidelines informs us:
Use the “robots.txt” file on your web server to manage your crawling budget by preventing crawling of infinite spaces such as search result pages.

When your website search engine creates result page and if it’s visible for indexing then search bots will waste their time to index it and they wouldn’t process needed pages, it will entail increase of indexing time and some of your good pages will be ignored. If you want to limit indexing then you should use “Disallow” directive.

No matter what we want, there are many details and situations just like in the SEO case when this advice is not optimal.

A lot of websites including big companies ignore this advice and grant access to their search result pages to the crawler bots. It really can make sense with the right approach — if search results which Google shows to your visitors correspond with their search requests and satisfy their needs then it could be useful for some types of websites.

Be careful. Your website could receive a Google penalty and get a low rank. CleanTalk doesn’t recommend to do it.

Quite possible that search result pages of your website will be not the most optimal ones which you desire to have.

Changing directive to “Disallow” alone is not enough to solve the problem of spam requests.

Spambot or a visitor searched something on your website using a spam phrase with a spam link and search result page will contain the phrase with the link even if are no pages found on your website.

The page will look like this:
Your search for “rent yacht here www.example.com” did not match any entries.

If your search result page is visible for indexing then crawler bots will know that your website gives links or mentions about such topic, therefore goal of a spammer to promote something is fulfilled and your website has necessary phrase and link (in some cases search result pages could have an active link).

To get rid of this problem you have already added “Disallow: /search” in your “robots.txt” file but this directive doesn’t fully forbid indexing and visiting these pages by crawler bots. Google tells us about that directly:
A robotted page can still be indexed if linked to from other sites
While Google won’t crawl or index the content blocked by “robots.txt”, we might still find and index a disallowed URL if it is linked from other places on the web. As a result, the URL address and, potentially, other publicly available information such as anchor text in links to the page can still appear in Google search results. To properly prevent your URL from appearing in Google Search results, you should password-protect the files on your server or use the noindex meta tag or response header (or remove the page entirely).

Thus you have to add NoIndex meta tag to your search result page template.

Google recommends:
To prevent most search engine web crawlers from indexing a page on your site, place the following meta tag into the section of your page:
<meta name="robots" content="noindex">

To prevent only Google web crawlers from indexing a page:
<meta name="googlebot" content="noindex">

You should be aware that some search engine web crawlers might interpret the NoIndex directive differently. As a result, it is possible that your page might still appear in results from other search engines.

Why it has to be done?

In a way you can call it a vulnerability and spammers use it for their own purposes. They search something on your website with needed key words then grab the link of the search results and copy-paste it to other web resources.

When Google bots visit your pages that have such link they follow it they land on the Disallowed page. But it doesn’t mean to stop indexing, so they index pages with spam search results.

As a result users who would search for the same phrases in Google might get pages with spam. It’s dangerous because some important data could be compromised such as phone numbers, contact e-mails and so on.

Load on Your Website via Search Form

How it works: your website has a search engine and visitors can input a word or a phrase they want to get information about. Search engine generates result pages and these pages are being visited by crawler bots, Google, Bing and the like. There could be dozens or even hundreds of pages of the search results, it could create a significant load on your website as your website generates a new result page every time. Spambots can use your search engine to perform a DDoS attack and your web server has to process a lot of actions.

So, how can you avoid these problems?

1. Add “Disallow” directive to the search result page.
2. Add tag NoIndex to the search result page template of your website. Be careful, make sure that other pages don’t have such tag or else Google will stop indexing them.
3. Set the limit of requests a one IP could have to use your search form.

All this is doable by yourself but we offer to use our anti-spam solution.

CleanTalk Anti-Spam has the option to protect your website search form from spambots.

1. Spam FireWall blocks access to all website pages for the most active spambots. It lowers your web server load and traffic just by doing this.
2. Anti-Spam protection for website search forms repels spambots.
3. Additional option can add NoIndex tag to forbid indexing.
4. If your search form gets data too often the CleanTalk plugin will add pause and increase it with each new attempt to send data. It saves your web server processor time.
5. Spam protection allows you to not forbid indexing for the crawler bots if you really need it but simultaneously you will get protection from spambots.
6. CleanTalk allows you to see what requests users did in the search form and what they were looking for. This will help you optimize your site and make information more accessible.

You can download CleanTalk Anti-Spam Plugin from WordPress Plugin Directory.

Note: Adding tags to search results pages will be added in one of the next releases. We will inform you.

Spam protection for search form is available for WordPress, Joomla 2.5, Drupal 8.

Update January 24, 2023
Search bots that visit pages with search results are not displayed in the anti-spam log as they do not carry useful information about spam bots or people visits.

Thank you!

About 735,000 IP addresses were returned to the registry. This is the first time that IP addresses have been taken from fraudsters after a trial.

On May 14, South Carolina U.S. Attorney Sherri Lydon filed criminal wire fraud charges against Amir Golestan, alleging he and his Charleston, S.C. based company Micfo LLC orchestrated an elaborate network of phony companies and aliases to secure more than 735,000 IPs from the American Registry for Internet Numbers (ARIN), a nonprofit which oversees IP addresses assigned to entities in the U.S., Canada, and parts of the Caribbean.

“Fraud will not be tolerated. The vast majority of organizations obtain their address space from ARIN in good faith according to the policies set out by the community. However, ARIN detected fraud as a result of internal due diligence processes, and took action to respond in this particularly egregious case,” said John Curran, ARIN President and CEO. “We are stepping up our efforts to actively investigate suspected cases of fraud against ARIN and will revoke resources and report unlawful activity to law enforcement whenever appropriate.”
https://www.prnewswire.com/news-releases/arin-wins-important-legal-case-and-precedent-against-fraud-300849070.html

According to a press release by ARIN, “Micfo obtained and utilized 11 shelf companies across the United States, and intentionally created false aliases purporting to be officers of those companies, to induce ARIN into issuing the fraudulently sought IPv4 resources and approving related transfers and reassignments of these addresses. The defrauding party was monetizing the assets obtained in the transfer market, and obtained resources under ARIN’s waiting list process.”

This case is also interesting due to the fact that according to some sources the IP addresses were resold to spammers.
This data Spamhaus The Powerhouse Network / IP.Gold

Statistics of CleanTalk Anti-Spam service about spam activity AS53889 Micfo, LLC.

Data provided on May 23, 2019.
https://cleantalk.org/blacklists/as53889

As we see, the IP addresses from AS53889 were used not only for sending email spam but for sending spam to web sites.

Since May 2018, spam network activity was small, an average of about 400 IP addresses were added to the blacklist. But in March 2019 spam activity increased dramatically and there were already almost 21,000 IP addresses in the blacklists.

Unfortunately, this case is not based on spamming, but only on obtaining IP addresses fraudulently. We hope that in the future, registrars will be able to conduct investigations and with the massive use of addresses to send spam and other malicious activity.

In spam statistics CleanTalk AS53889 is not the most spam active network.

We offer you a review of the top 10 most spam active networks. Data collected on May 23, 2019.

Top 10 Spam  IP Networks

1. IP Network 27.152.0.0/13
   This network belongs to AS4134 CHINANET FUJIAN PROVINCE NETWORK and has 524,286 IP addresses and currently 29,737 IP addresses in blacklists.
   Spam statistics for AS4134 CHINANET FUJIAN PROVINCE NETWORK
    
2. IP Network 79.184.0.0/13 This network belongs to AS5617 Orange Polska Spolka Akcyjna and has 524,286 IP addresses and currently 16,579 IP addresses in blacklists. Spam statistics for AS5617 Orange Polska Spolka Akcyjna
   
   
3. IP Network 49.64.0.0/11
   This network also belongs to AS4134 CHINANET FUJIAN PROVINCE NETWORK and has 524,286 IP addresses and currently 15,779 IP addresses in blacklists.
   Spam statistics for AS4134 CHINANET FUJIAN PROVINCE NETWORK
   
   
4. IP Network 14.160.0.0/11
   This network belongs to AS45899 VNPT Corp Vietnam and has 2,097,150 IP addresses and currently 12,382 IP addresses in blacklists.
   Spam statistics for AS45899 VNPT Corp Vietnam
   
   
5. IP Network 36.248.0.0/14
   This network belongs to AS4837 CHINA UNICOM China169 Backbone and has 262,142 IP addresses and currently 11,963 IP addresses in blacklists.
   Spam statistics for AS4837 CHINA UNICOM China169 Backbone
   
   
6. IP Network 117.24.0.0/13
   This network belongs to AS4134 CHINANET FUJIAN PROVINCE NETWORK and has 524,286 IP addresses and currently 11,255 IP addresses in blacklists.
   Spam statistics for AS4134 CHINANET FUJIAN PROVINCE NETWORK
   
   
7. IP Network 155.94.128.0/17
   This network belongs to AS8100 QuadraNet, Inc. and has 32,766 IP addresses and currently 10,249 IP addresses in blacklists.
   Spam statistics for AS8100 QuadraNet, Inc
   
   
8. IP Network 107.173.128.0/17
   This network belongs to AS36352 ColoCrossing and has 32,766 IP addresses and currently 9,785 IP addresses in blacklists.
   Spam statistics for AS36352 ColoCrossing
   
   
9. IP Network 95.79.0.0/16
   This network belongs to AS42682 JSC ER-Telecom Holding Russia and has 65,534 IP addresses and currently 9,567 IP addresses in blacklists.
   Spam statistics for AS42682 JSC ER-Telecom Holding Russia
   
   
10. IP Network 120.32.0.0/13
    This network belongs to AS4134 CHINANET FUJIAN PROVINCE NETWORK and has 524,286 IP addresses and currently 15,779 IP addresses in blacklists.
    Spam statistics for AS4134 CHINANET FUJIAN PROVINCE NETWORK

Full statistics on the spam activity of all autonomous systems you can see here https://cleantalk.org/blacklists/asn

The article used materials
https://krebsonsecurity.com/2019/05/a-tough-week-for-ip-address-scammers/

CleanTalk has updated the API method “spam_check” that allows checking spam activity of IP and email addresses via the CleanTalk database.

What’s new:

1. Displays the code of the country where the IP address belongs.
   This will help you know the geo-location of each IP address. The country code is displayed in a two-letter format in accordance with “ISO 3166-1 alpha-2”.
   
2. Added checking email addresses for existence. You can find out more about checking email addresses for existence here: Real-Time Email Address Existence Validation.
   
   Be careful, email is checked for existence only if you send only one email address in an API request. See API instructions.
   
3. Check email address for one-time use. Temporary email addresses for registration/comments and messages is one of the attributes of spam. What is wrong with using temporary email addresses? You can never contact this client and your marketing will not work. If this user forgets to log in/out, he will never be able to recover them.

Please, look at our API “spam_check” guide.

As everyone knows, WordPress sends a notification about a new comment to an article only to the author of the article. This is often inconvenient, as the site can be administered by a group of people and important messages may be lost, you need to enter the admin panel of the site to view them and etc.

CleanTalk Anti-Spam allows you to add user groups to receive notification of new comments. So, when a new comment to an article appears, the notification will be received not only by the author of the article but also by all users in the groups that you added.

See the instruction on how to add WordPress user groups to receive notifications for new comments.

We’ve launched the option to store the visited links of your visitors before they posted something on your website. It also includes the source where they came from to your website. (For the WordPress Plugin Only)

To enable or disable the option go to settings of the CleanTalk Anti-Spam Plugin. Go to your WP Dashboard —> Settings —> Anti-Spam by CleanTalk —> Advanced settings —> “Store visited URLs”
https://cleantalk.org/help/anti-spam-log#stored_URLs

“Store visited URLs” — the plugin stores the last 10 visited URLs (HTTP REFERRERS) and URL sources before your visitor submits data with your website form. You can see the stored URLs for each visitor in your CleanTalk Anti-Spam Log.

Help with website analytics — stored URLs could be used to your website analytics or to detect click fraud.

Additional Control
Spammers can fake or actually visit any webpages to pass through anti-spam protection. Take into account other factors such as if there are links, contacts, spam activity on other websites. You can check spam activity with the CleanTalk Database or with your Anti-Spam Log, you’ll see the number of spam attacks performed from IP address or email.

Enable this option to improve anti-spam protection.

If you have any questions, we will be happy to help you.
You can leave a comment below or create a private ticket here.

We have launched the option to add additional email addresses to get Weekly Anti-Spam Reports.

This is necessary for customers who care about receiving notifications for other site administrators or webmasters.

Use the option “Grant” to add additional email addresses.
Email notifications — allows adding other email addresses to receive Weekly Anti-Spam Report. 

If you want the report to come not only to you but also to your other employees, use this option to add email addresses.

Learn more, how to grant the rights.

CleanTalk has added a new function in their WordPress Security Plugin. With CleanTalk Security you will always know about any PHP errors on your website.

Are you sure that your website doesn’t have PHP errors?
Not all hosting companies enable PHP Log by default and you need some time to enable it and sometimes it looks difficult if you don’t have enough experience.

Why is it important?

Any PHP errors tell you that some of your website functionality doesn’t work correctly, furthermore hackers may use these errors to get access to your website.

So, CleanTalk WordPress Security Plugin collects PHP errors and sends them to your CleanTalk Dashboard.

PHP Log contains data/time and the type of error:
NOTICE
WARNING
FATAL_ERROR
UNKNOWN

Each type has a short comment, what does it mean and our recommendations for how to resolve it. You can view your log in CleanTalk Dashboard.

CleanTalk provides a simple and easy way to control all PHP errors and to prevent problems for your customers. You can enable this option on the settings page of CleanTalk WordPress Security plugin. Go to your WordPress Dashboard->Settings->Security by CleanTalk->General Settings->Miscellaneous and pick the option “Collect and send PHP logs”.

If you have any questions, we will be happy to help you.
Leave your comment below.