Search the Internet for Bible References on a Topic

Scraping a Web Site for Bible References

Using the function from the prior post, we can leverage PowerShell to download a web page and parse it for us to find references to scripture.

$requestResult = Invoke-WebRequest -Uri "https://blogs.lcms.org/2018/loving-your-internet-neighbor" -DisableKeepAlive -UseBasicParsing -ErrorAction SilentlyContinue

if ($requestResult.StatusCode -eq 200)

{

    Get-BibleReferences $requestResult.Content | Format-Table -AutoSize

}

Output

Reference    Book    Chapter Verses From To

---------    ----    ------- ------ ---- --

1 Cor. 1:23  1 Cor   1       23     23     

1 PETER 3:15 1 PETER 3       15     15     

John+5:39    John    5       39     39     

John 5:39    John    5       39     39     

Luke 24:27   Luke    24      27     27      

Search the Internet for Bible References on a Topic

So let's take this a bit farther.  Let's say someone sends you an email wanting to know more about the Biblical position on, let's say, heaven and hell.  So you could fire up Google and spend the next half hour looking at the first ten posts on heaven and hell.  But really, we should take what scripture says first, then consider the opinions of others.  This lead me to wanting to look at only the Bible verses relating to Heaven or Hell.

I have built a function which will query Google, read the first page of results then download each of the results, scrape them for Bible references, and return not only the reference but also the Bible text in NET Bible format.  Although it takes about a minute to run it saves a bunch of time!

Get-BibleReferencesOnTopic "Lutheran Heaven Hell" | Select-Object Reference, Text | Export-Csv -Path "References.csv" -NoTypeInformation

Sample output (400 Bible References):

The code is available on Github!