Content marketing in the age of NLP and BERT

Technological innovations like voice-based digital assistants have changed the ways that users search for and interact with content. Search engines like Google have sought to adapt to these changes by upgrading their algorithms to produce more relevant and user-friendly search results. Google has achieved this with its recent unveiling of BERT.

What is BERT?

BERT, short for Bidirectional Encoder Representations from Transformers, is Google’s relatively new neural network technique for natural language processing (NLP) pre-training. BERT was the product of research on transformers conducted by Jacob Devlin and his colleagues from Google in 2018. Transformers are models that process words in relation to all the other words in a sentence, as opposed to looking at words one-by-one in sequential order. BERT is the most significant change in search since Google released RankBrain in 2015. BERT was rolled out in October of 2019 for English-language queries; it was then expanded to 72 languages in December of the same year. Here’s a history of Google updates from Moz.

How does BERT work?

Computers have never been good at understanding language. To improve their comprehension, researchers turned to NLP. Some common NLP models include named entity recognition, classification and question answering. Yet a significant limitation of NLP models is that they are each only fit to solve one specific language task. The limitations of these common models are overcome by BERT. BERT can essentially handle all NLP tasks. Researchers developed BERT through a technique known as masking which involves hiding a random word in a sentence. BERT looks at the words before and after the masked word to predict what that word is (this is called a bi-directional model). Through repetition, the system gets better at predicting what the masked word is as well as understanding language in general. In essence, BERT helps train cutting-edge question and answer systems.

Why should we care about BERT?

The purpose of BERT, or any Google algorithm, is to help users find relevant and useful content. For instance, Google provides an illustration of search results before and after BERT for the search phrase “2019 Brazil traveller to USA need visa” (sic). Google points out that the word “to” is particularly important to understanding the meaning of the phrase. The traveller is looking to go from Brazil to the US, not the other way around. RankBrain and earlier Google algorithms were unable to understand the context of “to”, resulting in also returning results for US citizens travelling to Brazil. BERT can, however, pick up on the nuance of “to” and will only return results for users looking to go to the US. Google estimates that BERT will initially help Search better understand one in ten searches in the US in English and impact ranking for featured snippets. (Featured snippets--selected search results that are featured on top of Google's organic results below the ads in a box--are the most desired real estate in search rankings.) With so much at stake, marketers must change the ways that they write content and closely monitor the changes Google and other search engines apply to their algorithms to ensure that their online assets are employing the latest best practices for optimal visibility results.

How to rank well with BERT

Following the implementation of BERT, it is more important than ever that marketers write quality content. In the past, marketers attempted to rank well by stuffing a number of high-value keywords into a piece of content. This search engine optimization (SEO) method produced results in terms of ranking but significantly impacted readability. With BERT, this kind of poorly written content will have difficulty finding its way to the top of the search rankings. This is not to say that you should not continue to employ SEO best practices. You will still want to adhere to foundational SEO techniques including researching and transcreating keywords (particularly long-tail keywords), internal and external links, headings, etc. But you will want to use these strategies in a way that does not sacrifice the quality of your content. So, how do you go about writing good content to rank well in the BERT era? Start by focusing on the following six tactics.

Match questions to answers exactly

When crafting an answer, think of a one- or two-sentence response that Siri or Alexa would give that exactly matches the question. The basic format for answering a question should be [entity or subject matter] is [answer]. Imagine a user enters “where is Disney World?” The entity or subject is Disney World. Your content should include a simple answer to this question in the suggested format. So, “Disney World is located in Orlando, Florida”.

Identify units and classifications

Pay attention to the meanings of words and if they imply specific units and/or classifications. NLP will look for these when determining if content contains the answer to a question. For example, consider the query “temperature to cook pork chops”. Fahrenheit and Celsius are both units of temperature, so the answer must include the number and unit of measurement. So, a viable answer is “Cook all raw pork chops to a minimum internal temperature of 145°F”. Note that if you were writing for an audience in Europe, you would provide your answer in Celsius.

Get to the point

While long-flowing speech may sound nice and even have some artistic quality to it, BERT does not like it. One of the most important things that you should do is answer a question as clearly and concisely as possible.

Use keywords naturally

We mentioned above that keyword stuffing impacts readability and gets you penalized by Google. With BERT, you really need to get rid of that practice. Instead, use single keywords like you would in a normal conversation; they must not sound forced. Long-tail keywords have excellent SEO value, but they must be used in a way that caters to voice-based queries.

Use related keywords

Related keywords are those that commonly appear alongside a specific term. Using related keywords improves the relevancy of the target keyword. These terms should be identified prior to writing and sprinkled into content naturally. You are not trying to rank for these related terms, but to improve the recognition or understanding of the target phrase. For example, if you were trying to rank for “content marketing”, you could include related terms like SEO, SERP, word count, meta tags and so on to make the page more relevant.

Answer all aspects of a question

Get into the habit of following a query all the way through to answering follow-up or additional questions a user may have. In other words, your content should strive to answer all questions that a user could pose when they conduct a search. And as mentioned before, the more precise and useful your content is, the better it will rank. For example, if the user searches for “checking account”, you may also want to address “types of checking accounts” and “how to open a checking account”.

Pay attention to the usage of certain words

In the Brazil-to-US traveller example that we mentioned above, we highlighted the significance of the word “to”. Some additional words you want to pay attention to include but, not, except, from, in and about. With BERT, the meanings of these words matter; make sure you are using them in a way that is consistent with the intent of your content. BERT and other NLP technologies will continue to change the ways that Google and other search engines rank content. An increased emphasis will be placed on the user experience by returning search results that are more conversational and relevant. To optimize performance with BERT, it is more important than ever for marketers to deliver high-quality content. You don’t want to be behind the curve compared to your competitors. Get started by following the tips mentioned in this blog post and continue to monitor the use of BERT and other NLP models to stay up to date on the best content marketing practices. Thanks so much to Hinde Lamrani, our international search subject matter expert, for her help with the insights in this blog post!