Anti-phishing landing page: turning a 404 into a teachable moment

The Anti-Phishing Working Group (APWG) anti-phishing landing page [1] is a web page designed to be displayed in place of a phishing website that has been taken down. The page carries a succinct anti-phishing training message. The landing page is currently being used by financial institutions, ISPs, phishing site take-down vendors, government organizations and online merchants. When would-be phishing victims attempt to visit a phishing website that has been taken down, they are redirected to the landing page, hosted on the APWG website.

In this article, we describe the development of the landing page and present our analysis of the data we collected from its log files during the first six months of the landing page programme. Our analysis suggests that approximately 70,000 users were educated by the landing page during this period. We identified 3,917 unique phishing URLs that had been redirected to the landing page. We found 81 URLs in our log files that also appeared in email messages archived in the APWG phishing email repository. We present our analysis of the features of these emails.

In the past, when ISPs and registrars were asked to disable a phishing site, they would remove the site from the Internet. This meant that a user would see a 404 error when they tried to access the site. These 404 errors would often confuse users who believed they were visiting a legitimate website. Because of this, APWG and Carnegie Mellon decided to create an educational landing page.

The landing page is a web page containing an educational message to help consumers protect themselves from phishing. The page is hosted by the APWG. When ISPs and registrars are contacted about disabling phishing sites they are now asked to redirect all traffic attempting to access the phishing site to the landing page. This way, when users attempt to access the phishing site, instead of encountering a 404 error, they are taken to a page that educates them on how to protect themselves against phishing (See Figure 1).

Figure 1. APWG landing page. Users are presented with a version of the PhishGuru intervention when they click on a link to a phishing site that has been taken down.

The landing page approach is compelling for several reasons. First, it takes advantage of an ideal ‘teachable moment’ in that it directs training to the users who need it most – those who have ‘fallen’ for a phishing scam. In addition, the landing page enables users to be trained without taking time out of their busy schedules, and it motivates them to pay attention to the training. The landing page makes use of research results from PhishGuru, another programme aimed at educating users about the perils of phishing [2]. Finally, use of the landing page creates a repository of data that can be analysed to gain a better understanding of phishing.

The APWG landing page is based on the PhishGuru embedded training approach developed at Carnegie Mellon University. PhishGuru is an embedded training system that teaches users to avoid falling for phishing attacks by sending them simulated phishing emails. Figure 2 presents one version of the PhishGuru intervention. People access these training emails in their inbox when they check their regular email. The training emails look just like phishing emails, urging people to go to some website and log in. If users fall for the training email – that is, if they click on a link in that email – we provide an intervention message that explains that they are at risk of falling victim to phishing attacks and which offers tips they can follow to protect themselves. The training materials present the user with a comic strip that defines phishing, offers steps the user can follow to avoid falling for phishing attacks, and illustrates how easy it is for criminals to perpetrate such attacks. Our previous user studies in the laboratory and in the real world have validated the effectiveness of the PhishGuru approach [3], [4].

Figure 2. The final intervention design that we used in a large-scale real-world study [4].

We designed the landing page using a user-centred iterative design process. Our goal was to design a succinct and engaging training intervention that could be translated into multiple languages and formatted for a variety of devices, including handheld devices. We began by compiling suggestions for training content from members of the APWG IPC (Internet Policy Committee). While this design incorporated all of the content committee members wanted to include, there were concerns that it was too long and not clear enough for non-experts to understand. We developed a second condensed version that omitted some of the content that was not directly related to phishing and which shortened the phishing-related content. We then conducted two focus group studies to evaluate both the short and long versions of the proposed landing page and compare them with one of our PhishGuru cartoons.

The first focus group was a two-hour session at Carnegie Mellon University with nine participants of varying ages and educational backgrounds. Using a wall projector, we began by demonstrating how someone might click on a link in a phishing email and arrive at the landing page. We then showed them what they might see on a landing page. We discussed details of three versions of the intervention: (1) the committee draft; (2) a condensed draft; and (3) the PhishGuru cartoon. We provided participants with a colour printout of the designs and asked them to provide feedback. Participants said the committee draft and the condensed draft were both too long, and that they would not read the entire content of either. However, they were more positive about the PhishGuru version and said that they would be more likely to read its entire content. After listening to participants make many comments about how their grandparents would react to the landing page, we decided to determine how the landing pages would appeal to older people. To that end, we conducted a second focus group study.

The second focus group study was a 2.5-hour session with six participants at The Jewish Community Center of Greater Pittsburgh. We worked with AgeWell’s Independent Adult Services Department to recruit participants who were over 65 years old. Once again, participants came from a variety of backgrounds and none of them knew what phishing was. The process for conducting the study was similar to the first one. Using feedback from the first focus group, we revised the condensed and PhishGuru versions of the landing page. This time we discussed details of three versions of the intervention: (1) the committee draft; (2) the revised condensed draft; and (3) the revised PhishGuru.

Participants in this study, like those in the first, responded negatively to the committee draft. Most of the participants said they would not read the complete page. Participants liked the fact that the revised condensed version was short and had less text, but some participants mentioned that, even though it was shorter than the committee version, it was still too long for them to read in its entirety. Participants were attracted to the PhishGuru version, stating that it was fun to read and that people of all ages would read it. Participants were interested in the cartoon format and characters and said that they would read the complete intervention. All participants agreed that having cartoon characters is likely to attract readers’ attention.

In order to make this an industry-wide initiative that any organization could use, a publicly available sub-domain was set up on the APWG website. Information about the project was posted on this website. The English version of the landing page was hosted on the same website. Since this page was intended to be translated into many other languages, it was decided that users would be redirected to a specific language depending on the default language of their web browser. As of March 2009, volunteers had come forward to translate the landing page into Arabic, Bulgarian, Catalan, Danish, Dutch, French, German, Hebrew, Japanese, Korean, Romanian, Spanish and Swedish. The French landing page is already live [5].

The APWG’s server access log records all requests in Apache’s combined log format. By mining the landing page log files, we can create a list of phishing URLs that are redirected to the landing page. We correlated the log data with the APWG’s feed of reported phishing emails (emails sent to [email protected]) to find out which emails led most users to visit the landing page. This provided us with an insight into which phishing emails users are most likely to fall for. In the following sections we present an analysis of the logs we collected and results of feature analysis performed on the emails retrieved from the feed.

The data that we collect in the log files does not represent the entire population of users who click on the links in the phishing emails. If a user clicks on a link in the email and that link is already in the blacklist of the user’s browser, then access will be blocked and the user will not be redirected to the landing page. Also, ISPs and registrars eventually stop redirecting users to the landing page some time after a site has been taken down. If users click on such a link after the redirection has been removed, the user will be presented with a 404 error page. Thus, our data is a good lower bound for people who click on links in phishing emails.

We believe the landing page has created many teachable moments in which users have been trained to avoid falling for future phishing attacks. From the entire data, there were 78,541 total hits on the page; among these hits, 3,917 unique phishing URLs were redirected to the landing page. These statistics suggest that the landing page has been responsible for at least 71,504 ‘teachable moments’, in which a user has had the opportunity to learn from the intervention.

We observed that most hits (85.9%) came from the United States. This may be due to the fact that, at least for the time being, the brands that are requesting redirection to the landing page are mainly from the US. It also may be because the organizations being phished are mostly from the US. This may change as more brands around the world start using the landing page.

To study the emails that correspond to the phishing URLs being redirected to the landing page, we compared the unique URLs from the landing page logs to the URLs in the APWG email feed. We found 81 matches for the period from 1 October 2008 to 31 March 2009. We examined the 81 emails manually and analysed the features in these emails. Around 95% of the messages masqueraded as emails from one particular financial institution. The rest were made to look as if they were from other popular financial institutions and government agencies.

Most of the emails had features similar to legitimate messages. Ninety-one per cent of them had some form of logo or banner at the top. As researchers have shown, the fact that these logos and banners look legitimate is one of the main reasons people fall for phishing emails. Seventy-three per cent of the emails had some sort of footer containing logos; in particular, Bank of America emails had an Olympics logo in the bottom right-hand corner (see Figure 3). In some cases, phishers used an exact replica of a legitimate email. Figure 3 presents a legitimate email and a similar phishing email found in the AWPG feed.

Figure 3. Top: Phishing email from the APWG email dump that claims to come from Bank of America. Bottom: A real email from Bank of America to its customers. (All information with ‘%’ is used to customize the emails with personal information).

In this article, we discussed a real-world implementation of a landing page, based on PhishGuru, that educates consumers on how to avoid phishing attacks at the most teachable moment. Many users were educated by seeing the landing page instead of a 404 error page.

Since most phishing emails replicate legitimate emails, we believe that researchers and industry could reap substantial benefits by creating a corpus of legitimate emails, studying their features, and incorporating these features into email filters. Phishing emails haven’t changed much over time, remaining relatively unsophisticated and containing a great number of errors in grammar and formatting. Most of the emails in the log analysis asked users to click on a link in the message to update their account details.

Going forward, we plan to study the changes to the data as the landing page is deployed in additional languages and as more brands request redirection to the page.

A more detailed description of this research can be found at http://www.ceas.cc/papers-2009/ceas2009-paper-37.pdf.