Some people eat, sleep and chew gum, I do genealogy and write...

Sunday, September 25, 2016

Confronting the complication of genealogy online


Genealogy is a complex subject and when you add the complexity of the Internet, you may despair of ever understanding even the basics depending on whether your strengths lie in the area of genealogical research or computer and Internet technology or both. An interesting study, recently released by the Pew Research Center entitled, "Digital Readiness Gaps" addressed the following issues:
The analysis shows there are several distinct groups of Americans who fall along a spectrum of digital readiness from relatively more prepared to relatively hesitant. Those who tend to be hesitant about embracing technology in learning are below average on the measures of readiness, such as needing help with new electronic gadgets or having difficulty determining whether online information is trustworthy. Those whose profiles indicate a higher level of preparedness for using tech in learning are collectively above average on measures of digital readiness.
The study classifies 52% of the population of the United States as "Relatively Hesitant" and 48% as "Relatively more prepared." Only 17% of the population constitute the "Digitally Ready" who are confident in using digital tools to pursue learning. The study focused on "online learning." What makes this study relevant to online genealogy is that the entire subject of genealogy involves nearly constant learning and much of that is available today online as opposed to more traditional methods.

It is also significant that the demographics shown in the study of the "Unprepared" with relatively lower levels of technological adoption focused on women, 50 years and older, who make up the majority of the genealogical demographic. There is also a direct correlation between a person's level of education and their digital readiness. The most digitally ready group and those with higher income and education who are in their 30s and 40s. This is group is not the mainstay of genealogy.

The Pew Research Center study makes the following interesting observation about digital readiness:
More recently, as the internet and smartphones have spread through the population, adoption across population segments has been uneven. Pew Research Center recently reported that some users are unable to make the internet and mobile devices function adequately for key activities like looking for jobs. Communities face similar issues. Though the use of technology in schools has unfolded well in some places, problems have arisen where insufficient planning failed to take into account whether there was acceptance of new technology by all parts of the educational ecosystem. 
These examples underscore two points about how new technology works its way through society: First, different people and institutions have varying levels of preparedness for using next-generation technologies. Second, this reality can result in varying levels of usage of new technologies as they diffuse in society. These differences can, in turn, ultimately raise the possibility that uneven adoption and use of technology could have negative consequences for those whose are not facile and comfortable technology users.
Assumptions that any one segment of our society is either digitally advanced or inhibited depends more on the person's individual educational level and the person's degree of trust of online sources than it does a person's access to technology. These conclusions directly correspond to my own personal observations.

Although the study did not address younger Americans, my own observations are that blankly considering the "youth" as being both digitally prepared and able to absorb complex subjects online is a mistake.  For example, I have been teaching classes and doing presentations on genealogical subjects now for many years. From my observations, the number of people under the age of 20 who have attended any of my classes is so small as to be nonexistent and in reality, those under about the age of 50 are relatively rare. I have no way to know how many really young people read blogs or watch webinars, but I suspect that the number reflects their physical presence in classes.

So, until we can start helping the older members of the genealogical community to have a greater acceptance of learning online, particularly about genealogy, and the sooner we can help the youth learn about genealogy, the better off the entire community will become. Until both of these happen, we will maintain our marginal percentages of real participation in genealogical research.

Saturday, September 24, 2016

Record Hints vs. Manual Searching: Which wins?


All four (Ancestry.com, FamilySearch.org, MyHeritage.com and Findmypast.com, or five, if we include Geneanet.com) of the big, online genealogy databases provide algorithmic, programmed record hints. However, most genealogists, especially those with advanced technical savvy, rely more upon manual searches than they do the automated ones. Of course, in order to use the record hints, the genealogist must have a substantial family tree in each of the five programs and in my experience, even genealogists with considerable research experience balk at the idea of maintaining four or five separate databases, even assuming that the researcher does not have a separate, off-line database. Since the record hinting software uses the information contained in a family tree, those without family trees in each program do not benefit from the advanced technology.

What is interesting to me is that even if the researchers have their family trees in one or more of the programs, they still focus on manual searching even when those searches are not overly productive. I guess my question is what makes the researchers think they can out perform the automated record hints with manual searches on the same program. Commonly, I observe researchers who are manually searching one of the large database programs when they have a long list of waiting record hints to process.

Before getting further into this issue, it seems to me to be a good idea to examine the limitations of both manual and automated searches. First the limitations they have in common, some of which are pretty obvious.

Limitations on both manual and automated searches:

  • All database searches are limited to the content of the records in the database. This may seem elementary, but how many times do you check to make sure that the website has the type of record you are looking for before you make your search and of course, if the records are not in the database no automated process will make them appear. This could also be a controlling factor in choosing to put a family tree on a website.
  • The accuracy of the initial assumptions made by the researcher about the identity of family members, i.e. if you input the wrong parents, the program will not know this and will look for what it is told to look for.
  • The assumptions made by the algorithms programmed into the automated searches and the manual searches. 
  • The detail contained in the record sources. If the records contain little data about the target individuals, the searches will produce more false positives.
  • The accuracy of the indexing done to the original records.

Limitations on manual searches:

  • A manual search is usually limited to the information known and input by the researcher.
  • The searchers ability to formulate search terms. This is one of the primary issues. 
  • The time spent by the searcher.
  • The number of false positives returned by the search. Many searchers give up when the search engine begins producing long lists of obviously inappropriate responses.

Limitations on automated searches:

  • The program must rely on the accuracy of the information the user enters into the target family tree.
  • The depth of the information considered by the searches. This is one of the main factors limiting the first iterations of automated searches. Innovations in the past few years have increased the search depth and thereby increased the accuracy of the searches. 
  • The number of records included in the automated record hint program. 
  • The input from the user in the form of accepting or validating the record hints provided. In some cases, the user declines a valid record hint that will then inhibit the ability of the program to provide future hints.
  • Inability to 

From the researchers' standpoint, the automated record hints are a bonus. They provide information that would have cost the researcher time and effort. Maintaining the family tree on the website is the unavoidable overhead to obtaining adequate record hints.

I have written down some of the limitations, but what about advantages. From my own perspective, I find the record hints to be highly useful in accelerating my ability to fill in routine-type research. I regret all the hours I spent searching through U.S. Census records for example. Now, I get Census input from almost all of the programs. I think it is an amazing development and I am grateful for all the work that has gone into developing such a useful technology. I think that any researcher that doesn't take advantage of the technology is basically wasting their time.

Friday, September 23, 2016

Using Smart Technology to Jump-Start Your Genealogical Research: Part Eleven

Comprehensive mapping and gazetteer programs that open windows into historical locations

A Topographical Chart of the Bay of Narraganset in the Province of New England, with all the Isles contained therein, among which Rhode Island and Connonicut have been particularly Surveyed, Shewing the true position & bearings of the Banks, Shoals, Rocks &c. as likewise the Soundings: To which have been added the several Works & Batteries raised by the Americans. Taken by Order of the Principal Farmers on Rhode Island, By Charles Blaskowitz. Engraved & Printed for Wm. Faden, Charing Cross, as the Act directs, July 22d 1777. To the Right Honourable Hugh Earl Percy, Baron Percy, Lucy, Poinings, Fitz-Paine, Bryan & Latimer, Lieutenant General of His Majesty's Forces in Great Britain and America. This Plan is ... most humbly Inscribed, by ... Wm. Faden. (in manuscript) Hinck S.
Maps are more than and interesting sidelight to genealogy. All the recorded events in our ancestors' lives took place at a specific location on the  earth. Any records that may have been created to record the events are directly associated with that location. In genealogy, any geographically based area that had people with some official capacity for recording records is referred to as a jurisdiction. For example, the birth of a baby born today in Utah is recorded by a law that mandates the creation of a birth certificate by the State of Utah. That officially created birth certificate will be required for the child to be enrolled in school. In this case the jurisdiction for creating birth certificates is the state of Utah. That same baby will also be required to have a social security number shortly after birth. The jurisdiction for social security is the United States Federal Government.

These levels of government can be viewed as piling up like pancakes on a plate. The top pancake represents records created at the national level, the next level, here in the United States, is the state, then the county, the municipality and then personal records. At each level, there is someone who is interested in creating a record or is compelled to do so. 

Maps help us understand how those different jurisdictional levels were constituted at the time the event occurred. Of course, the records that were created may have been removed from their original place of creation. Genealogists spend most of their time trying to identify and locate these records that were created about our ancestors. This task becomes increasing difficult as time passes and the records are lost or destroyed. Eventually, if we go far enough back in time, there are no more reliable records about our ancestors. 

In order to find the records, we need to know certain things about our ancestors. Here is a summary of the information we need in order to locate records.
  • We need to know at least one very specific location where an event occurred in our ancestors' lives.
  • We need to know the general time frame when the event occurred.
  • We need to know the organization of the various jurisdictions that could have created the record, as they existed at the time of the event.
  • We need to know where the records are stored and available for examination today.
If we are missing any part of this information, we are really guessing about the identity of our ancestors. Maps and mapping programs are often essential in working through the historical changes that could have occurred. Political and social boundaries change over time. In the United States as the the population expanded, city, county and state boundaries changed. The names of geographic locations were changed or modified. The genealogical researcher has to determine how all of these various "jurisdictions" from nation governments to local school districts were constituted at the time our ancestors lived in order to understand where the records might have been created and subsequently stored. 

One important fact about the new digital technology is that the entire face of the earth as now been digitally mapped at a very high degree of accuracy and detail. In addition, hundreds of thousands of old maps have been digitized and are readily available online. There are also extensive, online, searchable lists of geographic place names that assist the researching in locating where events occurred. 

One of the most comprehensive programs for finding old maps is the OldMapsOnline.org portal program that indexes over 400,000 old maps with an efficient interface for viewing them. There are so many online mapping websites today that any attempt at making a list would be no more than a sample of the total number. For example, if I do a Google search for "maps New England old," I get millions of results. 

One of the most comprehensive collections of maps in the United States is maintained by the U.S. Geological Survey (USGS.gov).  The mapping division, includes The National Map with almost endless resources for viewing current and historical maps of the United States. Most important of these resources for genealogists are the Historical Topographical Map Collections. The viewer for these maps includes the ability to overlay different historical maps and view the changes over time. There are comprehensive instructions for using all of these websites on the websites. 

The USGS also maintains the United States Board on Geographic Names that was established in 1890 and has accumulated an immense online, searchable database that contains information on current and historical names from the United States and around the world. 

Identifying an exact location associated with your ancestor's life is just the first step in the research process. Finding the associated records is the real work of genealogical research and of course, then searching those records for information about your ancestors, evaluating the information and then systematically recording it is the product of our research. 


Thursday, September 22, 2016

New Videos added to the BYU Family History Library YouTube Channel


Finding and Merging Duplicate Records in the Family Tree - Kathryn Grant


Land Rights: Grazing, Timber and Mineral Rights and Leases for Genealogists - James Tanner


Training with the Family History Guide - Bob Taylor


The Anatomy of a Court Case for Genealogists - James Tanner

The Brigham Young University Family History Library is continuing to sponsor an increasingly broad offering of genealogy videos on the BYU Family History Library YouTube Channel. If you take a moment to subscribe to the Channel, Google will send you notices of the new videos as they are posted.

Using Smart Technology to Jump-Start Your Genealogical Research: Part Ten

Using Billions of digitized source documents in thousands of archive websites: Continued

The real challenge of doing genealogical research is locating records that contain information about our ancestors. Our modern technology has expedited finding those records that have been digitized and extensive indexing projects, both volunteer and commercial, have further enhanced the ability of researchers to extend the number of records that are discoverable. But the vast bulk of the remaining records are in repositories, each with its own unique cataloging system and catalog. In many cases, these catalogs are still paper-based and only accessible by physically visiting the location of the records. In many cases, records are stored but uncataloged and only physically searchable. The existence of such records may be discoverable online, but the examination process requires travel to the location of the records. 

Perhaps the largest of these repositories that still requires physical examination due to a lack of digitized records and adequate catalogs is the U.S. National Archives and their individual branches scattered across the country. The inherent limitation of the National Archives' catalog and all such catalogs is that they are inherently limited in their ability to provide the researcher with information about the specific content of the records. Cataloging systems provide general research assistance but always require direct examination of the items contained in the catalog. Digitization of those records enhances the availability of the contents, and optical character recognition or OCR further assists the researcher and indexing speeds up the searching process. 

The flood of online digitized documents often gives the impression that "everything is online." But any diligent genealogical researcher will soon find it necessary to go beyond what has been indexed and begin looking for even more information. Technology can measurably assist even those researchers that find themselves looking for paper records. One major impediment to much of the necessary research is the U.S. Copyright law. Copyright restrictions have created a huge body of genealogically valuable books and other records that can only be identified through catalogs and are not usually digitized. Researchers must either purchase the items or find them in a library. Fortunately, we have a huge, consolidated, online catalog to assist us in finding records, including books in tens of thousands of libraries across the world: the WorldCat.org.


With over 2 billion catalog entries from thousands of libraries worldwide, the WorldCat.org catalog is easily one of the top websites assisting researchers in finding revelevant books and other other items. Here is a screenshot of the WorldCat.org advanced search page.

A search for the term "genealogy" shows the depth of the catalog with over 1.3 million responses. The WorldCat.org website is, afterall, a library catalog and if the user enters his or her zip code, the program will provide a list of libraries that have any item selected. 

The WorldCat.org's sponsoring organization, the OCLC or Online Computer Library Center, Inc., has other valuable websites, including a huge online catalog of archive holdings in the United States called the ArchiveGrid.org

The ArchiveGrid.org website catalogs over 4 million records of archival material in over a thousand archives around the United States. For diligent researchers that look beyond what they can immediately view, the holdings of these archives can become a treasure trove of information and in some cases, may be the link to the only copies of some relevant documents.

Of course, there are many other online portals that collect links to vast collections of documents. In the United States, one significant effort in this regard is the Digital Public Library of America or dp.la


For genealogists, in addition to the links to huge numbers of relevant books, documents and other valuable resources, the Digital Public Library of America is partnering with FamilySearch.org to add access to the over 300,000 digital books in the FamilySearch.org Book collection.

It is impossible to do more than hint at the huge number of available records in the world. But it is important to understand that although many of these records are not easily researched online, there are a number of very valuable tools for finding those that are not online.

Here are the previous posts in this series.

Wednesday, September 21, 2016

Let It Be: Obsession and Genealogy

This image pretty much sums up my own brain except for the major part that is consumed with genealogy. Lately, however, I have been "helping" several genealogical researchers that seem to be obsessed with finding a particular ancestor or event in an ancestor's life. In one case, the patron at the Brigham Young University Library was consumed with discovering the location of an immigrant ancestor. Another patron was focused on the death date of an ancestor that died in the late 1800s to early 1900s somewhere in the United States. It seems like I have a steady stream of people who abandon all other avenues of research to focus on one date or one person. I am beginning to think that we might need some genealogical counseling services.

Quoting from the following article on Nature.com:

Obsessive–compulsive disorder: an integrative genetic and neurobiological perspective
David L. Pauls, Amitai Abramovitch, Scott L. Rauch & Daniel A. Geller Nature Reviews Neuroscience 15, 410–424 (2014)
An individual with obsessive–compulsive disorder experiences exaggerated concerns about danger, hygiene or harm that result in persistent conscious attention to the perceived threat or threats; in other words, they result in obsessions. In response to the distress and/or anxiety associated with these obsessions, the person acts (that is, performs a behaviour) to neutralize the distress and/or anxiety, which provides temporary relief from the anxiety associated with the obsession. However, this relief leads to reinforcement of the behaviours, leading to repetitive, compulsive behaviour when obsessions recur.
From my limited contact with the researchers, I can't possibly determine if their obsession with a specific research issue spills over into other obsessive behavior patterns, but I suspect that it might.

Let me approach the case of the person who is obsessed with finding a death date for an ancestor with some hypothetical solutions to the problem. First of all, and most importantly, not every event in every person's life has been recorded. Records are pervasive and there are many more than can possibly be imagined by the average researcher, but genealogical research does have its limits. Given the time period of the late 1800s or early 1900s, death records may or may not exist. Depending on the part of the United States where the person might have died, which pretty much covers the entire country, consistent recording of death dates and death records in general may not have been done as late as the 1920s in some areas of the country. So, the simple answer to this particular researcher's unending quest is that there is no record of the death of this particular person and continued searching will not reveal further information.

It is entirely possible, given the time period involved, that the elusive ancestor died somewhere unexpected. For example, transportation was readily available in the late 19th and early 20th Centuries in the U.S. and the ancestor could have gotten on a train, for some reason, and died in an unexpected place across the country. Perhaps, feeling ill, the person got off a train somewhere in the Midwest and died suddenly without anyone around who knew who the person was. Of course, I can continue to imagine all sorts of possibilities that would explain the missing death date and place.

But why does the researcher, who probably has better things to worry about, spend so much time looking for that one date? Is there any doubt that the person, who was born in the early 1800s is still alive? Not really, but this simple fact seems to be lost on the obsessive researcher.

What about the immigrant? This is a little more complex situation. Locating the place of origin of an immigrant is crucial in extending that particular family line. In the most recent cases I had to deal with, the researchers' ancestors came from Ireland, Germany and England and in none of these cases was there any more specific information than those general places. By the way, all of the elusive immigrant ancestors had extremely common names. The particular obsession of these recent patrons consisted of an inability to understand that research had to begin with searching for information here in the United States, rather than jumping immediately to records in Europe. After hours of trying to explain the need to do further research on the individual after arrival in this country, I had to give up trying.

It seems easy to say things like, try looking at a broader spectrum of records rather than returning to the same collection of U.S. Census records, but in one case the researcher had photocopies of the U.S. Census records and every time I tried to explain why we could not do research in Germany, the patron pulled out the records to show me where the Census said that the ancestor was born in Germany. The tragedy of this particular issue was that the ancestor was born in the early 1800s and there was no information about which of the German states or countries was actually the place of origin.

In this particular type of situation, I guess I just have to feel sorry for the researchers and hope that they move on to more productive pursuits. Genealogy can be a challenge but it is important to maintain a balance between obsession and reality. Ancestral lines do end at some point. It is also true, as I have personally found, that some ancestral mysteries can take decades to resolve. But it is dangerous to abandon all other genealogical research to focus on an issue that does not seem to ever be resolved.

In the case of searching for the immigrant, the patron clearly needed a lot more knowledge and experience to make any headway. In the case of the missing death place and date, the researcher just needs to accept the fact that the date and place are missing and get on with other research.

Tuesday, September 20, 2016

The Challenges of Adding Sources to Family Trees -- Part Three: Moving Sources Across Family Trees


The proliferation of online programs that provide helpful record hints connecting users to useful and genealogically reliable source documents has created some secondary challenges. Each of these programs require the user to have his or her family tree on each website. If you want to take advantage of several of these programs, you can have up to half a dozen or more separate copies of your family tree to maintain.

Because each of these family tree hosting programs use their own proprietary formats, there is very little consistency between them. Each program has its own criteria for creating source citations and storing copies of the pertinent documents. If you try to keep up even a moderately sized family tree on more than one program, you could spend nearly all your time trying to keep the different copies synchronized.

We are essentially faced with massive amounts of data that is isolated by each of the programs with ineffective methods of transferring that data where it can be collected into one centralized program. For example, a user can have literally thousands of record hints and be forced to move them individually to consolidate them into one program. Some of the existing programs have a method to share multiple sources at one time for a single individual or to add one source at a time to multiple individuals. Of course, some of the programs can only move one field of information at a time even within the program itself. No matter which of the limited ways prevail in the programs, the user is confronted with either ignoring huge numbers of hints or spending huge amounts of time merely transferring the information, sometime one field at a time.

Genealogists who want to take advantage of the record hints in multiple programs either have to maintain multiple family trees or spend a huge amount of time moving the hints to their chosen, designated main database program. Part of my own challenge is illustrated by the above image of the notice of Record Matches, I have on the MyHeritage.com program.

 Is there a solution?

It now appears that there is some detectable movement towards a solution but ultimately the resolution of the entire problem is still quite a distance down the road. Since I last wrote about this family tree isolationist issue, some additional synchronization connections for some users have been established but even with the additional ability to transfer individual data and sources between programs or individuals in the same program, much of the content still defies transfer except by tedious copy and paste methods and what synchronization that can be done involves mostly moving one source or one individual's data at a time. Even within programs, the ability to attach the same document to more than one person is often limited. There are some notable exceptions, but this seems to be one of those features that is waiting for some increased technology or consideration by the developers.

Of course, as I have noted previously, it is not in the interest of "competing programs" to cooperate and provide methods for the convenient and easy transfer of data. If I have my family tree on two competing programs, this is viewed as a potential loss of business. However, from my perspective, I value each of the programs for their individual contribution to my overall goal of discovering my ancestors and I am perfectly willing to use and support both programs. In fact, the ability of the programs to share information with our chosen main family tree program is an important criteria for the use of the programs. I could give specific examples using specific programs but by doing so I might imply that the other programs were more efficient or desirable and this would not be the case. They are all almost equally obtuse.

How do I handle the situation? Unfortunately, I end up ignoring much of the problem. I realize I have thousands of unresolved record hints sitting in different programs but I do not have the time to review even a small percentage of them. Instead, I focus on my own selected research objectives and only use the record hints when they coincide with my specific objectives. Of course, I am still burdened by the time it takes to move any helpful data from one program to another, but I can control the time I spend adding sources or whatever by avoiding adding the records hints to random people in my family tree every time I am notified of new hints.

Most of the programs also try to involve me in collaborating or contacting my potential relatives. For some, this can be a huge benefit, but for me personally, it is merely another time consuming issue. If I see information supplied by others that pertains to me direct interests, I may interact with others even some relatives, but in almost all cases, I find that the information in other family trees is either copied from my own or lacking in substantiating sources.

Depending on the amount of information any particular user has available, adding more information such as from DNA testing may be beneficial or not. This is not a one-size fits all situation. DNA testing is a tool for resolving some specific types of problems and absent those problems, a DNA test is merely additional data that must be integrated into a user's existing data. I have been repeatedly asked if I am going to get a DNA test. My answer is simple, right now, I have no unresolved relationship issues that will be measurably assisted by a DNA test. If and when I discover an issue that can be resolved with a test, I will be glad to take one. Meanwhile, I do not see that adding more information and contacts from "potential" relatives is going to resolve any of the families I am presently researching. I guess the bottom line is why would I pay for more DNA based "research hints" when I am overwhelmed with the ones I already have?

You can read the previous posts on this subject here:

http://genealogysstar.blogspot.com/2016/09/the-challenges-of-adding-sources-to.html
http://genealogysstar.blogspot.com/2016/08/the-challenges-of-adding-sources-to.html