Custom Function Database - Part 5

Lately, here at ISO FileMaker Magazine we’ve been covering a lot about web services and using technologies like REST, cURL and JSON.

Well, that’s not about change with this video. We’re now needing to move forward with our Custom Functions database and make a connection to the web site where the custom functions are hosted.

We do this with a dedicated Custom Function which takes advantage of the Base Elements plugin to make HTTP requests. With the knowledge learned in our cURL video about request & response headers, we can connect to the BrianDunning.com web site and get the id values of any new custom functions.

Making this type of request means we are relying on code which can break at some point in the future, should the web site itself change. So, I talk about how to deal with this and how to parse the data in the most efficient manner.

Let’s scrape some web data and get it into our database!

AttachmentSize
CustomFunctions05.zip219.94 KB

Comments

Thanks Matt, Looking forward to the video on connecting to API.

Using bBox > Python > BeautifulSoup doesn’t always capture the full function text from html.

The problem arrises when the function contains a ‘<‘ symbol. This class of symbols aren’t properly converted to html entities within the function's web page on BrianDunning.com.

When BeautifulSoup encounters the misplaced symbols, it tries to accommodate by assuming them to be tag related. This results in a corrupt/inaccurate html tree structure.

The Python script returns the function text up to the first “<“.

This “Recapture” Filemaker script can be applied when needed.

# Sometimes the Python/BeautifulSoup capture of the function body fails.
# This can happen if the text contains reserved HTML characters that have not been converted to entities.

Set Variable [ $result; Value:Let ([
~content = Functions::html;
~match = ""; ~start; 1)
];
Let ([
~content = Middle ( ~content; ~start; ~end - ~start );
~match = ">";
~start = Position ( ~content; ~match; 1; 1 ) + Length ( ~match );
~end = Length ( ~content ) + 1
];
Substitute ( Middle ( ~content; ~start; ~end - ~start ) ; Char(10) ; "" )
)
)]
Set Field [ Functions::function; $result ]