- Bi-directional reading
The act of a machine reading left-to-right and then right-to-left and encoding the word based on the embedded meaning
- POS tagging
A Java-based software that tags fine-grained parts of speech, such as a noun-plural, verb, proposition, etc.
- Transfer language
When a machine learns a task and then what it just learned is reused as a starting point for the subsequent task. This process is referred to as fine-tuning the machine’s knowledge processing capabilities
In the world of SEO and content writing, we often try to find ways to optimize our language and word usage for page rankings. Our SEO analyses tell us how the authority on a given topic is writing, the types of language they use, and important keywords. Most of this is based on the way that Google’s PageRank algorithm works. However, as of [last week/depending on date of publication] (December 18th, 2019), Google has officially rolled out its BERT update. If you’re worried that this update will drastically affect your SEO optimization plan, well, stop worrying. According to Google, this update will make your life easier.
That is Danny Sullivan, Google’s search engine liaison, who tweeted back in October that BERT was nothing to be worried about. However, much of the community noticed drastic changes after the October update, including major swings in SERPs and traffic. So what is BERT and do we need to take this into consideration moving forward? Before we address this question, it’s important to nail down a few expectations around this article. First, since BERT is actually more about machine learning than what content strategists might be familiar with, we will not be providing an intro into the current knowledge around machine learning or the history of its development, for that matter, but we will be including a healthy amount of links for those curious. You can consider this to be an introductory guide for those wanting to get their feet wet about BERT. Since the development of machine learning research is quite interesting, and relevant to our jobs as SEO and content experts, we want to dive into the research around AI and machine learning at a later date.
This article is the first part of a four-part series that will address the following:
ERT is a natural processing language (NPL) based deep-learning algorithm that is designed to adapt to our language use, rather than the other way around. In essence, the algorithm is learning the dynamics and nuances of 70+ languages in order to better understand a searcher’s intent when using normal language. Or, as Google puts it: “This technology enables anyone to train their own state-of-the-art question answering system.” BERT uses a variety of machine learning models in order to apply relational contextual cues when searching for our queries. The three main models are bi-directional reading, POS tagging, and transfer language, but if you want to dive into the science behind it you can start here.
Let’s first break down these three models:
In theory, BERT should learn more about how humans converse through how we are searching online, which could potentially be a problem. If we have adapted our language so that a machine could understand us, how much have we fundamentally changed the way we speak? Accordingly, BERT is really designed for those search queries that are longer, more sentence-like, and the queries that place a lot of emphasis on “to” or “for” and other placeholder propositions. Here’s an example of how BERT is promising to do better for the search community:
Much of what BERT science uncovered was that humans are, most of the time, super vague. According to Stephen Clark, Deepmind scientists who left Cambridge University, “Ambiguity is the greatest bottleneck to computational knowledge acquisition, the killer problem of all-natural language processing.” And we get this. But, as part of a company that regularly focuses on keywords in online language, it’s also hard to remember how we used to search before we knew that keywords were important.
Many of us are still learning about BERT and waiting to see what it is capable of. Google seems to think that we can stop using so much keyword-y language. However, they know that BERT isn’t perfect and it will still mess up from time to time. At first glance, it seems that we can expand our search queries to be more relaxed than SEO is tailored for. Since BERT is placing meaning on every word in every direction, it’s still important to be concise and directive in our language. However, this isn’t something that we can necessarily control. Much of the sway that BERT has caused is a more open market. Instead of strong holding trends based on keywords, others online, including niche businesses, businesses with a smaller budget, and those with a variety of things to offer, might draw a more eclectic crowd.