Author: Alexander

  • Non-visual methods to protect the site from spam. Part 1. Statistics

    Part 1. What statistic says

    Non-visual methods to protect the site from spam suggest automatic analysis of data coming from the visitor. As more data is analyzed, the more fully and more accurately visitor can be defined and made a decision is he a spammer or not.

    Systems that analyze such data usually accumulate visitor data statistics and the judgments. We offer an overview of the statistical data collected by us (service to protect sites from spam CleanTalk).

    Here I purposely do not cite the data analysis of IP addresses on black lists. Without them, you can obtain enough data, analyzing only the contents of form fields and HTTP headers.

    I’ll review the data by text message, nickname and email address and HTTP headers and the audit results of JavaScript test.

    Analysis on these figures algorithmically very simple and not demanding to resources, so it can be used before other more resource-intensive inspections.

    The data reflect the real picture at the time of writing and made on the basis of our analysis of the current traffic (more than 2 000 000 requests per day). Data can be freely used in the analysis of visitors to your sites. I note that the judgment for each criterion separately is not true — the best result will be achieved with a comprehensive analysis.

    1. Message text

    Message text – it is certainly the main thing in the spam. Consequently, spammers will build their posts so that on several criteria, they are clearly different from normal messages.

    The following table shows the most, in my view, informative statistics.

    Message text settings (average values) Not spam Spam
    Number of links, pcs 1.47 4.27
    Number of contacts (phone, e-mail), pcs 1.72 6.38
    Form filling time, sec 177 8
    The ratio of the length of the message to the time of filling, symbols/sec 23.81 308.54

    Amount of links speaks for itself. The amount of contact information can also be said about spam. Form filling time and, as a consequence, the rate of posts set differ most strongly.

    1. The nickname of the visitor

    The nickname can also tell about a lot of things. Probable cause is the quality of the algorithms of generating names that spammers use.

    Parameters of nickname (average values) Not spam Spam
    Length, symbols 7.40 16.52
    The number of delimiters, pcs 1.89 3.80
    The number of digits, pcs 3.29 7.59
    The length of a continuous sequence of consonant letters (for Latin), symbols 3.61 5.90

    One of the tasks of the spammer is not stumble on an error that a user with the same name is already on the site. So the uniqueness of nicknames currently provided, according to statistics, in the forehead – length, insert delimiters and numbers. As a result, you get a lot of nicknames with a large number of adjacent vowels and consonants, with the latter more.

    1. Name in e-mail

    Everything said for nicknames true for the name in the email.

    Parameters of name in e-mail (average values) Not spam Spam
    Length, symbols 10.09 19.16
    The number of delimiters, pcs 1.62 4.12
    The number of digits, pcs 4.30 9.57

    Note that as the delimiters characters are often used point – generated character string, then it randomly adds points, so you get a lot of e-mail names.

    1. HTTP-headers

    Spam-bots forge their headers to not be very different from the browser.

    However, statistics show that this is often true only at the time of writing the bot. In the future, it continues to work and send clearly outdated titles that can be seen in the table below.

    The percentage of HTTP headers User-Agent Not spam Spam
    Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) 0.01% 11.42%
    Opera/9.80 (Windows NT 6.2; Win64; x64) Presto/2.12.388 Version/12.17 0.01% 10.84%

    Ready spam solutions may also leave their headings, in particular, when using HTTP-proxy. And this is also reflected in our statistics.

    The percentage of HTTP headers Via Not spam Spam
    Mikrotik HttpProxy 0.86% 33.07%
    1. JavaScript-test

    Additional simple but very effective check can be JavaScript-test. For example, changing the JS-code the desired cookies, the options are many.

    The most advanced (and expensive) bots pass JS-tests. However, as can be seen from the statistics, a large percentage of spam comes from very simple programs, unable to do so.

    Percentage of failing JS-test Not spam Spam
    change cookies through JS 0.41% 68.53%
    1. Conclusion

    I have shown statistical data collected by our system at the moment. Again, for the most accurate solution to spam/not spam you need to analyze the indexes comprehensively, as well as in combination with other methods of spam checks.

    Learn more about CleanTalk Anti-Spam.

  • Solve the problem with caching of dynamic JavaScript code on the frontend of WordPress

    In the process of developing anti-spam plugin CleanTalk for WordPress we faced with the problem of caching of dynamic JavaScript code on the frontend of sites. Namely, if you place JavaScript that contains any pieces of code that can be dynamically inserted from backend site, in the presence on the site of any plug-in caching pages, JavaScript code is not possible to use as directed.

    Consider the example

    In the backend we have the template of JavaScript code,

     <?php
    $html = '
    <script type="text/javascript">
    function ctSetCookie(c_name, value, def_value) {
     document.cookie = c_name + "=" + escape(value.replace(/^def_value$/, value)) + "; path=/";
    }
    ctSetCookie("%s", "%s", "%s");
    </script>
    '; 
    
    $ct_checkjs_key = rand(0,100); // The value of the variable dynamic
    $field_name = 'ct_checkjs'; // The value of a static
    $ct_checkjs_def = 0; // The value of a static
    
    $html = sprintf($html, $field_name, $ct_checkjs_key, $ct_checkjs_def);
    ?>
    
    

    An example of the output on the frontend,

     <script type="text/javascript">
    function ctSetCookie(c_name, value, def_value) {
    document.cookie = c_name + "=" + escape(value.replace(/^def_value$/, value)) + "; path=/";
    }
    ctSetCookie("ct_checkjs", "455", "0");
    </script>

    Accordingly, the cache gets the JavaScript code in which parameter value of function ctSetCookie unchanged on all pages of the site and the same for all visitors, which leads to the impossibility of using JavaScript individually for each visitor. Consider options for solutions.

    Use built-in tools to disable caching

    If the plug-in of caching of content on WordPress more or less popular, then it is bound to have a means to exclude a list of pages from the cache. For example, for the WP Super cache, you can specify in your plug-in code line,

    define( "DONOTCACHEPAGE", true );

    This will be enough for your pages with dynamic code were not included in the cache. The disadvantages of this approach,

    It is necessary to integrate and test your plug-in with popular caching plug-ins.

    Still there will be cases when your code incorrectly works off due to the fact that one or another site is set rarely used plug-in of caching.

    And most importantly, this approach virtually eliminates the use of caching plugins, if your JavaScript code is placed on all pages of the website, or on the most loaded pages.

    Let’s look at other option solutions.

    AJAX call to the backend

    The essence of this approach is that on the frontend place only a static JavaScript code, and all that is required to use dynamically obtain the backend of the site through an AJAX call. The example code on frontend,

    
    //
    // Making a call to admin-ajax.php
    //
    function sendRequest(url,callback,postData) {
        var req = createXMLHTTPObject();
        if (!req) return;
        var method = "GET";
        req.open(method,url,true);
        if (postData)
                req.setRequestHeader('Content-type','application/x-www-form-urlencoded');
            req.onreadystatechange = function () {
                if (req.readyState != 4) return;
                if (req.status != 200 && req.status != 304) {
                    return;
                }
            callback(req);
        };
        if (req.readyState == 4) return;
        req.send(postData);
        return null;
    }
    var XMLHttpFactories = [
        function () {return new XMLHttpRequest()},
        function () {return new ActiveXObject("Msxml2.XMLHTTP")},
        function () {return new ActiveXObject("Msxml3.XMLHTTP")},
        function () {return new ActiveXObject("Microsoft.XMLHTTP")}
    ];
    function createXMLHTTPObject() {
        var xmlhttp = false;
        for (var i=0;i<XMLHttpFactories.length;i++) {
            try {
                xmlhttp = XMLHttpFactories[i]();
            }
            catch (e) {
                continue;
            }
            break;
        }
        return xmlhttp;
    }
    
    //
    // Process the results of the AJAX call.
    //
    function ct_callback(req)
    {
    ct_cookie=req.responseText.trim();  
        ct_setCookie('ct_checkjs', ct_cookie);
    
    return null;
    }
    //
    // Set cookie
    //
    function ct_setCookie(name, value)
    {
        document.cookie = name+" =; expires=Thu, 01 Jan 1970 00:00:01 GMT; path = /";
        document.cookie = name+" =; expires=Thu, 01 Jan 1970 00:00:01 GMT";
    
        var date = new Date;
        date.setDate(date.getDate() + 1);
        setTimeout(function() { document.cookie = name+"=" + value + "; expires=" + date.toUTCString() + "; path = /;"}, 500);
    
        return null;
    }
    
    var ct_ajaxurl = 'http://wordpress.local/wp-admin/admin-ajax.php';
    sendRequest(ct_ajaxurl+'?'+Math.random(),ct_callback,'action=ct_get_cookie');
    

    Please pay attention to the structure

    ct_ajaxurl+'?'+Math.random()

    This approach is used to avoid caching including an AJAX call.

    Move to the last listing, look at the backend,

    
    add_action( 'wp_ajax_nopriv_ct_get_cookie', 'ct_get_cookie',1 );
    /**
     * Returns a new cookie
    */
    function ct_get_cookie()
    {
        global $ct_checkjs_def;
        $ct_checkjs_key = ct_get_checkjs_value(true); 
        print $ct_checkjs_key;
        die();
    }
    

    Disadvantage of this approach only in one thing – your plug-in does one call more in the backend of WordPress. Given the fact that the hosting service cannot be the fastest or the WordPress can be set more than a dozen plug-ins, such a call would increase the response time of the site.

    Good luck in developing for WordPress!

    Learn more about CleanTalk Anti-Spam.

  • Statistics for the personal blacklist

    Dear Customers,

    CleanTalk informs you that in the dashboard query statistics is available on the personal blacklist.

    You can track the number, date and time of all attempts to use forms on the website by users from your personal blacklist.

    To view the statistics, go to the dashboard CleanTalk https://cleantalk.org/my/ select the website and go to Settings->Personal blacklists.

    Remember that if you mark a request as Spam in the CleanTalk dashboard, the IP/email will be added to your personal blacklist.

    To view existing entries, you can follow this link https://cleantalk.org/my/show_requests?int=week

    If you have questions, we will be happy to answer them!

    Supervise your anti spam through Dashboard.
    47129519-860c-4492-989c-8f8d9c2fb83e
  • How to save resources and to attract more customers for hosting company.

    Spam FireWall is an additional option in the anti-spam plugin CleanTalk. The principle of operation SFW differs significantly from the mechanism of anti-spam plugins.

    CleanTalk anti-spam filters spam bots at the moment when they are trying to post a comment on the website, register, i.e. make a POST request. In some cases, the spam bots sends not only POST requests, but GET, that is downloads web pages and thereby overloads the site with requests. In small amounts it is imperceptible, but when the website receives 20 000 – 80 000 spam queries per day, it may affect the CPU usage.

    CleanTalk SpamFireWall allows you to block POST and GET requests to the website and does not load a page.

    The spam bot is given specially formed page.

    SpamFireWall will block most (about 99%) of requests from spam bots, because the base is formed of the most active spam IP addresses. The rest of the spam bots will be blocked by anti-spam service in the POST request.

    In addition to the spam bots will be blocked all HTTP/HTTPS requests from spam active IP addresses. It helps to block HTTP DDoS attacks, SQL attacks, scanning website, etc.

    If the clients of a hosting provider use SFW on their websites, it can significantly reduce the load on the CPU servers. This allows you to place on one server more websites, to make the cost lower or to increase profits. To offer its current and new customers additional free services included in the rate, and thereby increasing the value of hosting services and customer loyalty. Customers do not have to spend time searching for a working anti-spam solution and test it, they will get a proven solution with the hosting.

    The percentage of blocked requests usually from 7% to 30%, depending on the traffic of the website. The most appropriate use of the service on a shared hosting, as it gives greater efficiency due to a larger number of clients on the server.

    Hosting providers use on their servers some software solutions to block malicious traffic, but as a rule, used a database of IP addresses is incomplete or they do not always intersect with the database of IP addresses CleanTalk. Therefore, web sites customers are exposed to tens, hundreds and even thousands of spam attacks per day. In addition to spam attacks from these IP addresses, there are other types of attacks on websites bruteforce, hacking, DDoS, SQL injection, etc. CleanTalk also get the most current information about IP addresses and their activity through cloud technologies.

    As mentioned above, customers prefer to get all-in-one and not waste time looking for solutions. Offering customers hosting with anti-spam solution, you save them a lot of time to focus on the development of the project and business.

    Often the hosting provider does not see any requests or needs of the customer, because such questions accepted to solve on their own, my site – my problems, and hosting is just a place where the site is. In addition to saving time and resources of the client, you give them a solution to the problem, which concerns every site.

    Based on experience in the ISP, it is possible to compare the needs of customers. When using an ISP, we are often faced with the question, which turned out to be the cause of viruses on client computers, which have suffered due to both clients and network infrastructure. We chose an antivirus and started offering it to clients, we have included a free version in the price and have received tens of thousands of installations. Thus, we solved two problems: the problem of clients and problem with the load on the network. At the same time direct inquiries about an antivirus we have not received. It was an unconscious need that we have helped to satisfy. So it is with hosting customers, you can help them solve their problems and remove some of the load from the host.

    CleanTalk provides its clients work logs of the service so clients can monitor the entire process of protecting a website from spam.

    For each request there is a record containing all required parameters date/time, IP/email and text messages.

    The client can check the correctness of filtration, because he can view the text or by checking IP/email, whether recorded other attacks with these IP/email to other websites. Efficiency of determination spam bots is 99.982%. Pay attention to the reviews in the catalogs, there is not a single negative comment about the quality of anti-spam service.

    We do not offer to sell to hosting customers CleanTalk Anti-Spam, and include it as an additional free option in the tariff. Customers only have to install the plugin CleanTalk on the website and it will immediately work. CleanTalk provides tech support and customer service independently.

  • CleanTalk Anti-Spam Released a New Version of the Spam FireWall

    CleanTalk company Inc is a cloud service protecting websites from spam bots, has announced the launch of a new version of the Spam FireWall which is designed to block spam attacks on the web sites.

    The CleanTalk SpamFirewall manages and filters all inbound HTTP traffic to protect web sites from spam bots and to reduce the load on the web servers.

    Spam FireWall – allows blocking the most active spam bots before they get access to web site. It prevents loading of pages of the web site by spam bots, so your web server doesn’t need perform all scripts on these pages. Also it prevents scanning of pages of the web site spam bots. Therefore Spam FireWall significantly can reduce the load on your web server. Spam FireWall also makes CleanTalk the two-step protection from spam bots. Spam FireWall is the first step and it blocks the most active spam bots, CleanTalk Anti-Spam is the second step and it checks all other requests on the web site in the moment before submit comments/registers and etc.

    How Spam FireWall works?

    -The visitor enters to your web site.
    -HTTP request data is checked of the nearly 5,8 million of certain IP spam bots.
    -If it is an active spam bot, it gets a blank page, if it is a visitor then it gets a site page. This is completely transparent to the visitors.
    -All the CleanTalk Spam FireWall activity is being logged in the process of filtering.

    CleanTalk’s Spam FireWall Features

    -Protection from spam bots without access to the web site. Spam FireWall blocks most of the spam bots before they load the page of the website.

    -Reducing the load on a web server. In order to post spam, many spam bots load the page, this creates a burden on the database and the server, and when a large amount of spam attacks it can have a significant impact on the performance of the website.

    -Protection against HTTP/HTTPS DDoS attacks. This is one of the most common types of DDoS attacks with the aim to load a web server so that it was not able to handle all other requests.

    -Protection against RPC-XML attacks. One of the most common types of attacks on sites running WordPress in order to pick up the username and password of the administrator of the web site or to organize DDoS attacks. Spam FireWall’s SQL Protection provides an affordable, automated solution for protecting from a variety of SQL injection attacks.

    -Spam FireWall’s logs allows you to monitor the service work and reporting all incidents.

    -Installation for 60 sec does not require modification of configuration files and others.

    -Spam FireWall is available for web sites on WordPress and Joomla

    Spam bots messages (comments) often disguised as ordinary users posts, but contain advertising links or text. The main objectives of such messages are the translation of the user to a malicious resource, advertisement, or by the links to raise the position of their site. This compromises the site and can spoil the reputation, the search engines lower the position of the site in the search results. That is why reliable protection from spam bots is only way to prevent the undesirable effects of cyber attacks. CleanTalk provides reliable protection from attacks and spam bots and promotes strengthening information security throughout the world.

    CleanTalk Spam protection FireWall based on the use of private data black lists of IP addresses.

    The main consumers are the administrators and owners of web sites, the solutions offered by CleanTalk allows to obtain an effective and automated solution to many security problems of the web sites and to save time for business development.

    Another area of use is the use CleanTalk for hosting providers, as it can reduce the load on web servers to save resources and costs.

    About CleanTalk

    CleanTalk is a SaaS spam protection service for Web sites. CleanTalk uses protection methods which are invisible for site visitors. Connecting to the service eliminates needs for CAPTCHA, questions and answers and other methods of protection, complicating the exchange of information on the site. Their solutions are reliable, easy and efficient. The module is completely invisible to the visitors and allows you to permanently abandon the ways of protection that impedes the communication of visitors to the site (CAPTCHA, question-answer, etc.). CleanTalk allows you to automate protection against distributed from spam and registration spam bots.

    The team CleanTalk has been developing a cloud spam protection system for 4 years and has created a truly reliable anti-spam service designed for you to ensure your safety.

    CleanTalk

    CleanTalk Spam FireWall

  • Manage Personal Black/White Lists

    CleanTalk informs you about the occurrence of an opportunity to manage personal black/white lists. You can view, add, and delete their items in the Control Panel->Logs

    How it works

    Go to CleanTalk Dashboard
    Select web site and click “Logs”

    Each records has menu for manage, if you mark record as “Spam” – this IP/Email will be added in your personal BlackList and will be always blocked on your website

    If you mark record as “Not Spam” – CleanTalk will not check this IP/Email

    To view personal lists click “Personal blacklists” under the record.

    Here you can change status – just click on status and it will be changed. If you delete item then CleanTalk checks it as usually. You should delete or change status of both IP and email because if you delete only IP so that visitor will be blocked because his email is still in your personal blacklist.

    Also you can add other IP or email in your personal BlackLists or WhiteLists. Enter the necessary IP or email then select status and click Save.

  • CleanTalk is one of the fastest anti spam plugins

    Everyone knows about why the faster your site loads, the better your customer experience is, the higher your Pagerank will be, and the higher your site will convert. Speed is becoming increasingly important in Search Engine Optimization , conversion and user experience. Today, site speed is one of the most important ranking factors on Google. A site that loads slowly will lose visitors and potential revenue.

    Every plugin adds a bit of complexity to your site, and it’s important to install well-developed plugins from a reputable source.

    If one plugin is slowing your site down, you can find a faster one or remove it altogether. In general, we always want to have the fastest set of plugins possible for the features you want the plugins to add!

    There are different ways of improving your site’s loading performance, an important parameter for the performance site to install well-developed plugins from a reputable source.

    Among anti-spam plugins CleanTalk Anti-Spam is one of the fastest. Despite the large plug-in functionality, the developers manage to optimize the performance of the plugin so that CleanTalk faster than most analogs. This contributes to the cloud service architecture, as all calculations take place in the cloud, not on the server, the server receives the finished result for further action.

    About CleanTalk

    CleanTalk is a SaaS spam protection service for Web sites. CleanTalk uses protection methods which are invisible for site visitors. Connecting to the service eliminates needs for CAPTCHA, questions and answers and other methods of protection, complicating the exchange of information on the site. Their solutions are reliable, easy and efficient. The plugins are completely invisible to the visitors and allows you to permanently abandon the ways of protection that impedes the communication of visitors to the site (CAPTCHA, question-answer, etc.). CleanTalk allows you to automate protection against distributed from spam and registration spam bots.

    The team CleanTalk has been developing a cloud spam protection system for 4 years and has created a truly reliable anti-spam service designed for you to ensure your safety.

    PressKit https://cleantalk.org/presskit

  • How to protect your WordPress site against spam and spam bots

    How to protect your WordPress site against spam and spam bots

    There are many plugins to protect against spam, almost all of them have some disadvantages. In our view it is optimal to use the cloud service CleanTalk.

    Since this is a cloud service, by obtaining and analyzing data from over 100,000 web sites, CleanTalk very effectively protects against spam. The algorithms analyze the behavior of spam bots increase service efficiency up to 99.998%. This is one of the fastest anti spam plugins and does not load the server and database.

    To start use CleanTalk on your WordPress site, follow these steps:

    Go to WordPress Dashboard->Plugins->Add New and in the search bar, type CleanTalk and click Install.

    install CleanTalk

    Activate the plugin and go to settings CleanTalk.

    To connect the plugin to the service, you’ll need your Access key. To get the key click on the button “Get access key”.

    Get key

    You will be taken to the website CleanTalk. You can change your email to register for the service.

    Register for an account

    Push the button and get your access key.

    CleanTalk anti spam setup on WordPress

    Return to the plugin settings, insert the access key and click “Save Changes”. The installation and configuration of the plugin completed, changes in Advanced Settings needed in rare cases.

    To test the plugin, log out of the account administrator and go to your website. Write a test review or make a test registration with e-mail *@cl*******.org, these messages will be blocked.

    test message

    Next, you should get a message about blocking

    forbidden

    Great, your website protected from spam bots!

    Similarly you can check any form in your website.

    Additional features CleanTalk. Dashboard, view logs.

    To view service logs, go to CleanTalk Dashboard. Or log in to your WordPress Dashboard->Settings-CleanTalk and click “Click here to get anti-spam statistics”

    get stat

    If you have any questions you can always contact us. We will be happy to help you.

    For more info

    Help

    Features

  • Short statistics of six months 2015

    We decided to bring intermediate results of six months 2015.

    This is the results observed in spam attacks.

    The number of spam attacks prevented on the sites for CMS:
    WordPress 241 753 033
    Joomla 35 511 344
    phpBB 4 019 935
    Drupal 2 633 882

    Of six months 2015 CleanTalk prevented 331 395 138 spam attacks.

    As we observe the reduction in the average number of spam attacks per day on site for about 40-50% compared to 2014. Perhaps, this is due to the fact that spam bots can’t post spam on the site and don’t want to spend more time and resources.

  • Spam FireWall – how to reduce CPU usage on website and to block DDoS attacks

    Spam FireWall – how to reduce CPU usage on website and to block DDoS attacks

    The CleanTalk SpamFirewall manages and filtres all inbound HTTP traffic to protect web sites from spam bots and to reduce the load on the web servers.

    CleanTalk has got an advanced option “Spam FireWall” for WordPress and Joomla!, this option allows blocking the most active spam bots before they get access to web site. It prevents loading of pages of the web site by spam bots, so your web server doesn’t need perform all scripts on these pages. Also it prevents scanning of pages of the web site spam bots.

    Therefore Spam FireWall significantly can reduce the load on your web server.

    Spam FireWall also makes Cleantalk the two-step protection from spam bots. Spam FireWall is the first step and it blocks the most active spam bots, CleanTalk Anti-Spam is the second step and it checks all other requests on the web site in the moment before submit comments/registers and etc.

    How Spam FireWall works

    • The visitor enters to your web site.
    • HTTP request data is checked of the nearly 5,8 million of certain IP spam bots
    • If it is an active spam bot, it gets a blank page, if it is a visitor then it gets a site page. This is completely transparent to the visitors.

    All the CleanTalk Spam FireWall activity is being logged in the process of filtering. The logs will be available for viewing in CleanTalk Dashboard since 10/15/2015.

    Spam FireWall DDos Protection
    Spam FireWall can mitigate the HTTP/HTTPS DDoS attacks. When an intruder makes use of GET/POST requests to attacks on your website. Spam FireWall blocks all requests from the bad IP addresses. Your website will issue give for infringer a special page instead of the website pages. Therefore Spam FireWall can help to reduce of CPU usage on your server.

    Get SpamFireWall