LinkedIn’s Dark Matter – Undiscovered Profiles

Posted by | March 21, 2011 | Dark Matter, LinkedIn | 40 Comments

Sourcing has a fundamental problem: All searches return results.

Yes, that is actually a problem.

Why? Because everyone’s a winner.

Type in a few keywords and BAM! – you get some good looking results. Hey, this sourcing stuff isn’t so hard!

If I’ve said it once, I’ve said it a thousand times – sourcing is easy. In fact, it’s ridiculously easy to find some people.

So if you and your company are happy with finding some people and not necessarily the best people available to be found, then you can stop reading now and go back to finding some people.

For everyone who’s still reading this, try answering these questions:

  1. Can you ever be sure you’re finding everyone there is to be found?
  2. How do you know you’ve found the best people available?
  3. How do you know you’ve found all of the best people?
  4. Are there people on LinkedIn, in your ATS, in job board resume databases that are never found?
  5. How can you be aware of social media profiles and resumes that your searches can’t return in results – but are there?

Sourcing is easy, but it’s not easy to get to the point where you are sure you have found all of the best available results, nor is it easy to specifically target and find people others cannot and do not.

Most people use relatively basic, straight forward/direct keyword and title searches. There’s nothing wrong with that – they clearly “work” – anyone running those kinds of searches will find results.

However, they will also find exactly what everyone else finds when searching for the same types of people, which yields zero competitive advantage.

The fact that all searches produce results is a problem because it lulls people into thinking that sourcing is easy, and at least on the subconscious level – it leads people to believe that the results that are returned from searches represent all available matching and relevant results.

However, it is a fact that no single search can find all of the people you’re looking for, and there are many social media profiles and resumes that are never found.

Let me introduce you to the concept of Dark Matter.

The Dark Matter of Human Capital Data

People who use relatively straight forward/direct keyword and title searches find the same people as everyone else using the same keywords and titles.

What’s particularly interesting to me is the fact that everyone running basic, straight forward/direct keyword and title searches also fail to find the people who are there to be found, but cannot be found via straight forward, direct searches.

These profiles are what can be referred to as “dark matter” – which, in layman’s terms, is stuff that’s there, but you can’t see it and it isn’t directly detectable.

I’ve found that people have an especially difficult time understanding that every search they run both includes some of the right people, and excludes some of the right people.

My last LinkedIn sourcing challenge provides a good example of that. Many respondents could find matching results, but did not seem to grasp the concept of ensuring that their searches did not exclude viable, available results. To some degree, that’s understandable, because it’s difficult to be aware of what you cannot see. However, just because your searches didn’t return a result, it doesn’t mean they are not there to be found.

So what happens to the people that are excluded from your searches – the people your searches don’t return, the people you’re not even aware of?

Well, it should be obvious that very few people find them, and those that do typically do so by mistake and not through a direct effort to uncover them.

They are there for the finding if you can:

  1. First realize that they do exist, even without direct evidence of such
  2. Figure out how to target and retrieve them

Revealing LinkedIn’s Dark Matter Profiles

I’m going to use a few very basic examples in an attempt to clearly illustrate what I’m talking about with regard to profiles on LinkedIn that most people #1 aren’t even aware of, and #2 cannot be found via conventional search approaches.

For these examples, it is very important that you don’t get caught up on the actual search terms I use – the terms themselves are irrelevant. What is relevant is the concept behind what’s being demonstrated. It’s up to you to absorb the concept and apply it to whatever types of people you are trying to find.

LinkedIn Dark Matter Example #1

Let’s say you’re looking for a software engineer and your manager is very clear that she needs someone with strong Java/J2EE skills and experience.

Most people searching for software engineers with Java will search for “Java.” As obvious and innocent as that would seem, doing so creates dark matter profiles.

Click the image below to reveal nearly 40,000 people that cannot be found by searching with the term “Java:”

For those of you who do I.T. sourcing and recruiting, you will quickly identify that the vast majority of the people returned by the above search to be software engineers who quite obviously have Java experience, yet they do not mention “Java” – their primary development language – anywhere on their LinkedIn profile.

Why would a Java developer not explicitly mention “Java” on their LinkedIn profile?

I know why – do you?

LinkedIn Dark Matter Example #2

Let’s say you’re looking for accountants, and your manager tells you that they must be CPA’s.

Of course, you can find people by searching for all of the various ways someone can express being a CPA – (CPA OR “Certified Public Accountant”). You might even get crafty, realizing that LinkedIn sees “C.P.A.” as distinctly different than CPA and add that to the OR statement.

There are nearly 5,000 people in all of LinkedIn that cannot be found unless you specifically search for “C.P.A.” – here’s the proof.

However, searching for any variant of CPA is still a very basic and direct method of searching for people who are CPAs.

Click the image below to see over 10,000 people who are highly likely (I’d say > 90%) to be CPA’s, but cannot be found by searching directly for any mention of CPA:

After getting over 600,000 profiles returned from my initial search targeting a current or past title of “Accountant” and specifically excluding any mention of CPA, I filtered the results by people who specify that they currently work for either Deloitte, PwC, KPMG or Ernst & Young.

Do you think a “senior accountant” at PwC isn’t a CPA? :-)

Those results represent more than 10,000 accounting professionals who work for Deloitte, PwC, KPMG or Ernst & Young that simply do not exist to anyone searching for (CPA or “C.P.A.” OR “Certified Public Accountant”).

Why would a CPA not make explicit mention of it on their LinkedIn profile?

I know why – do you?

Darkest Matter

The darkest dark matter profiles on LinkedIn are those of people who use very few, if any words that give you any real clue as to their specific skillset or experience.

These profiles are often never found, because any obvious keyword will prevent them from being returned in search results.

To reveal and target these profiles, you have to exclude the more obvious keywords that most people who actually use.

For example, if I were looking for software engineers, but did not want to see the profiles that everyone else is finding, I could target a current title of  (“software engineer” OR developer OR programmer) and try to exclude most major technologies with -(“java/j2ee” OR peoplesoft OR perl OR oracle OR SAP OR java OR j2EE OR .Net OR C# OR C++) in the keyword search field.

There are almost 1,000,000 profiles returned by that search, check them out here.

There’s nearly 13,000 of them from IBM, 12,000 from Accenture, and over 4,000 from Google.

Any sourcer or recruiter who puts Java, PeopleSoft, Perl, Oracle, SAP, J2EE, .Net, C#, or C++ in their search is simply not aware that the nearly 1,000,000 people I just found with the above search even exist because they will be excluded from their search results.

Why would you want to look at profiles of software engineers who don’t mention any particular technology on their LinkedIn profile?

I know why – do you?

Indirect Search

One way of discovering Dark Matter profiles is to conduct a form of Talent Mining I refer to as Indirect Search.

There are many applications of Indirect Search methodology, including the 2 examples I shared above, and you can also see some in action among many of the proposed solutions to my Ruby LinkedIn Sourcing Challenge.

The challenge really was an example of Dark Matter profiles on LinkedIn – I asked readers to find people on LinkedIn who had Ruby on Rails experience, but did not mention any variant of Ruby on their LinkedIn profile.

Many people who solved the challenge had to go indirectly search for LinkedIn profiles by first going outside of LinkedIn to find information on people that alluded to Ruby experience,  and then cross reference the name on LinkedIn to see if the person’s profile did or did not mention Ruby.

While that is one form of Indirect Search – finding people on LinkedIn with specific experience by finding them first elsewhere – how could you find people with Ruby experience on LinkedIn who do not make any explicit mention of it on their LinkedIn profile, AND they don’t have any other info elsewhere on the Internet to give you easy clues indicating that they have Ruby experience?

Can Semantic Search Help?

No.

While some people feel semantic search solves all sourcing problems (vendors, mostly :-)),  semantic search isn’t the solution to the Dark Matter challenge, for a number of reasons – not the least of which is that semantic clustering and other applications of machine learning can only work (semi)reliably with what is actually there/mentioned.

Even the most advanced NLP algorithms are limited to “analyzing” and “learning” from text.

What happens if the text isn’t there to analyze and learn from in the first place?

While there are solutions that are going to become available that claim to use semantic search that can cross reference LinkedIn profiles with other social media profiles and help identify people who may reveal some information via Twitter or other networks that they do not on LinkedIn – that too only works with information that is directly retrievable.

As you can see even from the extremely simple examples I demonstrated above, it is actually necessary to think and purposefully construct indirect searches for implied experience or qualifications in order to reveal Dark Matter profiles – those that cannot be seen/found by direct searching methods. It’s also interesting to note that more powerful forms of Indirect Search require the strategic exclusion of direct search terms – something I am not aware that any semantic search application allows. Does anyone know of one?

Before I get flamed by the semantic search vendors and fanboys/girls – I’m passionate about semantic search, and I’ve actually used most of the “big name” solutions on the market. I also perform semantic search manually at the most powerful level (sentence level verb/noun combos) to help fill job openings just about every day – ask me how if you’re interested.

Because I really understand and appreciate information retrieval – both manual and automated – I feel it’s my duty to keep vendors who are selling semantic search as the solution to all sourcing/recruiting challenges honest and to inform people who may be considering purchasing solutions that there are limitations of semantic search.

I’ll be writing more on the subject very soon, but I’d like to illustrate another issue with “semantic search” solutions that vendors won’t volunteer: If 5 companies who are constantly battling it out for top talent all use the same semantic search application, all 5 companies will find all of the same people. That means no competitive advantage. Perhaps even more importantly, they’ll also be totally unaware of the people their semantic search solution could not find – a common hidden talent pool of Dark Matter profiles that cannot be tapped by any of the companies.

Dark Matter Doesn’t Discriminate

Dark Matter profiles don’t magically appear for people who pay top dollar for a LinkedIn Recruiter account.

However, one powerful feature of LinkedIn Recruiter that can put a small dent into the Dark Matter of LinkedIn is the “All Groups” search, which allows you to target people in any group even if you are not a member. At least one person who solved my Ruby LinkedIn Sourcing Challenge did so by using this feature. However, there is a Yang for every Yin – it won’t work for the large number of people who haven’t joined groups, or haven’t joined a group that you can use to reliably identify them.

Regardless of level of account, no one searching LinkedIn, in any way or method, is immune to the Dark Matter phenomenon – it is a fundamental challenge to human capital information retrieval. Quite simply, you can’t directly search for and retrieve information that isn’t there to be retrieved. There are many great people that can only be found via Indirect Search methods.

To be sure, the Dark Matter concept certainly isn’t limited to LinkedIn – it applies to every source of human capital data, including your ATS/CRM, Monster’s resume database, the Internet, etc.

LinkedIn Dark Matter Question for You

What percent of LinkedIn do you think can be classified as Dark Matter – profiles that are there, but are never found or viewed through direct search methods?

Related Posts Plugin for WordPress, Blogger...

About Glen Cathey

Glen Cathey is a sourcing and recruiting thought leader with over 16 years of experience working in large staffing agency and global RPO environments (>1,000 recruiters and nearly 100,000 hires annually). Starting out his career as a top producing recruiter, he quickly advanced into senior management roles and now currently serves as the SVP of Strategic Talent Acquisition and Innovation for Kforce, working out of their renowned National Recruiting Center with over 300 recruiters. Often requested to speak on sourcing and recruiting best practices, trends and strategies, Glen has traveled internationally to present at many talent acquisition conferences (5X LinkedIn Talent Connect - U.S. '10, '11, '12, Toronto '12, London '12, 2X Australasian Talent Conference - Sydney & Melbourne '11, '12, 6X SourceCon, 2X TruLondon, 2X HCI) and is regularly requested to present to companies (e.g., PwC, Deloitte, Intel, Booz Allen Hamilton, Citigroup, etc.). This blog is his personal passion and does not represent the views or opinions of anyone other than himself.

  • http://www.facebook.com/people/Scott-Simmons/718460401 Scott Simmons

    Very interesting comment – you have my head spinning this morning!

  • http://www.facebook.com/people/Scott-Simmons/718460401 Scott Simmons

    Very interesting information. I look forward to more.

  • Patrick Ryan

    Potential resources have caught on that LinkedIn is a Recruiters playground. Although they liked to be connected to all of their business friends, they are beginning to be annoyed by all the requests by Recruiters asking to connect or requesting a meeting to talk about a potential job. I’m suspecting that more and more profiles are gravitating towards less information or keywords to avoid being found by Recruiters. To answer your question, “What percent of LinkedIn do you think can be classified as Dark Matter…?” I’m guessing 80%

  • Srini

    I would like to reiterate Patricks’ comments. I have experienced the same – talented people are fading away from the usual searches as they tend to hide their skills and expertise on Linkedin profiles. I guess, it will be a time soon when only “indirect search” methods would bring you a pool of qualifiable talented professionals for your jobs from LinkedIn.

  • John Turnberg

    Quick comment on the CPA search. I filtered the CPA search for the San Francisco Bay Area and then check a few profiles against the CPA database and it turns out, only one of the first 5 were a licensed CPA in CA even though they were working at CPA Firms. In California, just changing the URL and scraping the data is an easier way to find people. Look at this sample list from the State of California:
    http://budurl.com/CAcpaListA – it loads really, really slow, but the it gives you a great starting point for your research.

  • http://www.booleanblackbelt.com Glen Cathey

    No doubt that going to a state’s license database is a more direct method of identifying people with a CPA. :-)

    Just trying to point out that there are some CPA’s who do not mention it on their LinkedIn profile, and cannot be found by searching for such.

    With regard to the CPA database you referenced, I found people from the SF area who do mention CPA on their LinkedIn profile and work for one of the Big 4 who cannot be found using the “License Search for Individual Licenses.”

    Here’s 2 from the first page::
    http://linkd.in/ik9sWH
    http://linkd.in/hDYUEx

    Thoughts?

  • http://twitter.com/Sarangbrahme Sarang Brahme

    Amazing insight on LinkedIn. The problem is people (read recruiters) consider this as a job board and search likewise. Also LI is leaving no stone unturned to picture and sound it as a job board.

    Reality check: People do not register on LinkedIn (at least everyone) to showcase himself / herself as a candidate. Hence, they do not talk about themselves in detail to list every skill / certification etc like resume/CV. Also, it’s proven that most of people do not visit LinkedIn every month. All in all – LINKEDIN IS NOT A JOB BOARD.

    Most of people leave their footprints and it is our duty to unearth this “Dark Matter”. As a recruiter – our approach to LinkedIn should be very different than that of a job board. Hats of and Kudos to Glen on his deep and strategic thinking. It’s a wake up call to every recruiter who thinks they gain a competitive advantage by using LinkedIn. Not anymore!!!! Thanks to LinkedIn Recruiter…

    A sourcer who can tap these “Dark Matter” profiles will be a real “Sourcing pro” as oppose to job board miners. In my opinion – around 70%+ profiles comes under “Dark Matter”. A very simple example…

    Headstrong is an IT Services company mostly (90%) provides services to Financial Services / Banks. It’s their bread and butter!!! Let’s search LinkedIn to find out how many people have added any keywords related to this industry who are working with Headstrong

    Advance Search:

    Title:engineer OR developer OR analyst OR consultant OR programmer
    Company: Headstrong
    Results: 1200+

    Now, let’s find out “Dark Matter” profiles
    keywords: -banking -bank -”financial services” -securities -bonds -investment -derivatives -equity -insurance -asset -murex -calypso (you can add many more)
    Title:engineer OR developer OR analyst OR consultant OR programmer
    Company: Headstrong
    Results: 900+

    In case I don’t know about Headstrong and want to find out developers with domain experience; I will typically use all industry keywords along with job titles. Guess what – I am missing out on 70% of good results.

  • Bwill

    Great article! I’m interested in finding out more about this – “I also perform semantic search manually at the most powerful level (sentence level verb/noun combos) to help fill job openings just about every day – ask me how if you’re interested.” Thanks!

  • Katie

    Hey Bwill! I am interested in anything you can share. Looking for an Operations Manager who has banking lockbox experience. Please email me at: perrykatie@charter.net

  • http://www.booleanblackbelt.com Glen Cathey

    Excellent example Sarang!

    I do agree that the % of Dark Matter on LinkedIn is quite high, and it’s fascinating to know that most people simply aren’t aware of how many available profiles they have access to that they simply cannot return by the way they structure their searches.

    They don’t know what they’re missing. Literally!

  • Ian Stammers

    I would have to disagree with Patricks comments. I commented on a LinkedIn pole recently which was asking whether people see recruiters in a positive or negative light. Nearly 90% of nearly a 1000 people said they welcomed recruiters contacting them as long as it was appropriate and they had done their homework. As a veteran recruiter I always look for new ways to find people and this article is very helpful. Patrick at the bottom of everyone’s profile you can tick whether you want to be contacted regarding career opportunities or consulting offers so if you choose not to be contacted by a recruiter then you should tick this box.

  • Deborah

    I LOVE this. Just when I thought I was seeing all that I could see on LinkedIn. Thank you!

  • http://www.booleanblackbelt.com Glen Cathey

    Ian,
    I am glad you brought that LinkedIn poll up – I remember it well.

    90% of employed people did not mind being approached by a recruiter as long as the opportunity was relevant.

    If someone has a minimalist LinkedIn profile, perhaps stripped down to companies and titles alone, with no supporting descriptions as to their responsibilities and the environments they’ve worked in – how can anyone really approach that person with a relevant opportunity?

    How can a recruiter know what’s relevant to someone based on incomplete information?

  • Pingback: Recruitment Marketing Articles of Week 3.19.11 to 3.25.11 « Recruitment Marketing Innovation, Technology and Ideas

  • Oliver

    Excellent info! You’ve opened my eyes to things I hadn’t thought about trying before. On your comment; “I also perform semantic search manually at the most powerful level (sentence level verb/noun combos) to help fill job openings just about every day – ask me how if you’re interested.” I’m interested and asking :) Anything you can share would be great and much appreciated. Contact me at oeaton@diversant.com
    Thanks!

  • Pingback: Sourcers and Recruiters – Don’t Fear Watson or Semantic Search

  • Pratishnaik

    Glen,
    Thanks for sharing. Very Insightful.

    Ian:
    I look forward to your info. Any other insights you can share regarding Semantic Search. Please forward to patrick@fepsearchgroup.com

  • Pingback: Beyond Boolean: Human Capital Information Retrieval

  • Pingback: Sourcing is an Investigative and Iterative Process

  • Pingback: Update Your LinkedIn X-Ray Searches for Location Names

  • Cvasu

    Glen,

    I’m also very interested in learning more about this – “I also perform semantic search manually at the most powerful level
    (sentence level verb/noun combos) to help fill job openings just about
    every day – ask me how if you’re interested.” Can you please e–mail me at cvasu@professionalsg.com?

  • Tawana

    Hi Glen!  I appreciate all of the wonderful information you put out here for us Sourcers/Recruiters.  I would also like to learn about Semantic Searches.  You can contact me at tgates@sherpallc.com.  Thanks!

  • Pingback: How to Search for Top Students and GPA’s on LinkedIn

  • Pingback: How to Search for Top Students and GPA’s on LinkedIn- Nice one to read « Venkatesh Kothapalli

  • Pingback: Are You a “Dark Matter” Job Seeker on LinkedIn?

  • http://twitter.com/TriciaGroom Patricia Groom

    I read this last night, and this morning saw it in action.  I stumbled across a job advert for a -get this- Programming Software Development & Support Consultant.
    Somehow me and him aren’t talking the same language.

  • Pingback: Sourcing is an Investigative and Iterative Process- Boolean Black Belt « Venkatesh Kothapalli

  • http://www.seoconsultant.ie SEO Consultant

    I just had a shortest interview in my life. It was a guy
    from LinkedIn. His first question was:

    What is a dark matter?

    I said I do not know. The interview was pretty much over.

    He did have a funny explanation what the dark matter is
    though: http://www.irishrecruiter.com/linkedin-recruiter-what-is-a-dark-matter/

  • Pingback: Why So Many People Stink at Searching

  • Pingback: Talent Sourcing: Man vs. AI/Black Box Semantic Search

  • Pingback: How to Effectively Source Talent via Social Media & Networks

  • Pingback: Top 15 Common Talent Sourcing Mistakes

  • Jessica

    Hi Sarang,
    I tried your search yesterday and the results were only from “Staffing and Recruiting” industry. When I start the search “All” is selected. Do you think you could help me figure this out?

  • Pingback: The Top 15 Sourcing and Recruiting Mistakes « Michael Chiang's Blog

  • Pingback: 100+ Free Sourcing & Recruiting Tools, Guides, and Resources

  • Hans

    “I know why – do you?”

    No, but I am trying to figure it out.

  • Pingback: Using Extended Boolean to Achieve Semantic Search in Sourcing | Boolean Black Belt-Sourcing/Recruiting

  • Pingback: Is Boolean Search Boring and Less Effective than Semantic Search? | Boolean Black Belt-Sourcing/Recruiting

  • John Childs

    Well spoken, Glenn! John Childs

  • Pingback: Free LinkedIn Sourcing Webinar Wednesday November 20 @ 2PM ET | Boolean Black Belt-Sourcing/Recruiting