Exercise: Calling Data from an API

A query is basically a question, so, let’s try to answer this question: How many ‘knights’ are in the department of Medieval Art of the Metropolitan Museum of Art? To answer that question, we have to segment it into multiple parts:

  1. What is required to search within a specific department in the API?
  2. What endpoint do we need to use?
  3. What parameters do we need to use?
  4. What’s the previous information we need to retrieve before doing the query?
  5. What is the query we need to use?
  6. How do we get the specific information we need?

Seems like a lot of steps, but it’s not too complex as it might seem. Let’s start with the first step: looking at the API documentation: Metropolitan Museum of Art Collection API

Because the first question is about searching within a specific scope, let’s start with the search endpoint. As we can see, there is a list of paremeters, the first one is q, which is the one that helps us to search by term (like ‘knight’); and scrolling down a little, we can see the departmentId parameter, which requires a numeric (integer) identifier corresponding to the department we want to search within:

Parameter Format Notes
q Search term e.g. sunflowers Returns a listing of all Object IDs for objects that contain the search query within the object’s data
departmentId Integer Returns objects that are a part of a specific department. For a list of departments and department IDs, refer to our /department endpoint: https://collectionapi.metmuseum.org/public/collection/v1/departments

As you can see, documentation is the best starting point to interact with an API. With that information, we have answered not only the first question, but also what endpoint, parameters, and previous information we need to retrieve before doing the query.

Before proceeding with the next question, let’s get the list of departments and their IDs to select the one corresponding to the ‘Medieval Art’ department. To do that, just paste the ‘/department’ endpoint as is pointed out in the documentation (if you get lost, follow the hints ;) ).

Scroll down to see the list of departments and their departmentId. Copy the value of the departmentId for the ‘Medieval Art’ department (the number, not the name). We’ve underlined the value in the list for you.

Code 
viewof departmentListEndpoint = Inputs.text({
    label: "Department list",
    placeholder: "",
    value: "",
    attributes: {
        class: "form-control mb-3"
    }
})

/**
 * The purpose of this function is only to validate the endpoint
 * for students. Have this in mind if want to replicate the
 * code in other exercises.
 */
async function validateEndpoint(endpoint, query, params) {
    const checks = [
        {
            pattern: /^https:\/\//,
            message: "URL must start with 'https://'"
        },
        {
            pattern: /collectionapi\.metmuseum\.org/,
            message: "URL must contain 'collectionapi.metmuseum.org'"
        },
        {
            pattern: /\/public\//,
            message: "Missing '/public/' in the path"
        },
        {
            pattern: /\/collection\//,
            message: "Missing '/collection/' in the path"
        },
        {
            pattern: /\/v1\//,
            message: "Missing '/v1/' in the path"
        }
    ];

    if (params) {
        const paramsList = params.split("&");
        checks.push(
            {
                pattern: new RegExp(`${query}\\?`),
                message: `URL must contain '${query}' before parameters`
            },
            {
                pattern: /[?]/g,
                message: "Query and parameters must be separated by '?'"
            },
            ...paramsList.map(param => ({
                pattern: new RegExp(param),
                message: `Missing parameter: ${param}`
            }))
        );
    } else {
        checks.push(
            {
                pattern: new RegExp(`${query}$`),
                message: `URL must end with '${query}'`
            }
        )
    }

    if (/[^:]\/\//.test(endpoint)) {
        return "Invalid URL: Contains double slashes (//)";
    }

    const segments = ['collectionapi.metmuseum.org', 'public', 'collection', 'v1', query];
    for (let i = 0; i < segments.length - 1; i++) {
        const pattern = new RegExp(`${segments[i]}[^/]+${segments[i + 1]}`);
        if (pattern.test(endpoint)) {
            return `Missing slash between '${segments[i]}' and '${segments[i + 1]}'`;
        }
    }

    const basePattern = new RegExp(`^https:\/\/collectionapi\.metmuseum\.org\/public\/collection\/v1\/${query}`);
    if (!basePattern.test(endpoint.split('?')[0])) {
        return `Base URL is incorrect. Should start with: https://collectionapi.metmuseum.org/public/collection/v1/${query}`;
    }

    for (const check of checks) {
        if (!check.pattern.test(endpoint)) {
            return check.message;
        }
    }

    return null;
}

async function fetchDepartmentList(endpoint) {
    try {
        const response = await fetch(endpoint);
        const status = {
            code: response.status,
            ok: response.ok,
            text: response.statusText
        };

        if (!response.ok) {
            throw new Error(`HTTP error! status: ${response.status}`);
        }

        const data = await response.json();
        return { data, status};
    } catch (error) {
        return {
            data: { "Message": `Error: ${error.message}` },
            status: {
                code: 400,
                ok: false,
                text: "Bad Request"
            }
        };
    }
}

departmentList = {
    if (departmentListEndpoint) {
        const validation = await validateEndpoint(departmentListEndpoint, "departments", "");
        if (validation) {
            return {
                data: { "Message": validation },
                status: {
                    code: 400,
                    ok: false,
                    text: "Bad Request"
                }
            };
        }
        const result = await fetchDepartmentList(departmentListEndpoint);
        return result;
    } else {
        return {
            data: { "Message": "You need to enter a valid endpoint." },
            status: {
                code: 400,
                ok: false,
                text: "Bad Request"
            }
        };
    }
}

prettyDepartmentList = {
    let jsonString;
    if (typeof departmentList === 'string') {
        jsonString = JSON.stringify(JSON.parse(departmentList), null, 2);
    } else {
        jsonString = JSON.stringify(departmentList, null, 2);
    }
    
    // Highlight the entire Medieval Art object
    return jsonString.replace(
        /(\{[^\}]*"displayName":\s*"Medieval Art"[^\}]*\})/g, 
        '<span style="background-color: #fff3cd; display: inline-block; width: 100%;">$1</span>'
    );
}    

viewof prettyDepartmentListContainer = {
    let content;
    if (departmentList.data.Message) {
        content = html`<div class="alert alert-warning m-0">${departmentList.data.Message}</div>`;
    } else {
        content = html`<pre class="card-body m-0" style="background-color: #f8f9fa; max-height: 400px; overflow-y: auto;">${prettyDepartmentList}</pre>`;
    }
    
    const badgeClass = departmentList.status.ok ? "bg-success" : "bg-danger";
    
    const container = html`<div class="card">
        <div class="card-header d-flex justify-content-between align-items-center">
            <span>Department list</span>
            <span class="badge ${badgeClass}">${departmentList.status.code} ${departmentList.status.text}</span>
        </div>
        ${content}
    </div>`;
    return container;
}

The next step will consist of answering What is the query we need to use to search for ‘knights’ in the department of Medieval Art?

As we’ve seen in the documentation, we will require two parameters: q and departmentId. The first one is the search term ‘knight’, and the second one is the departmentId we got from the previous step.

Therefore, write here the query that will retrive all the objects that contain the term ‘knight’ in the department of Medieval Art.

Code 
viewof knightsQuery = Inputs.text({
    label: "Query",
    placeholder: "",
    value: "",
    attributes: {
        class: "form-control mb-3"
    }
});

async function fetchQuery(endpoint) {
    try {
        const response = await fetch(endpoint);
        const status = {
            code: response.status,
            ok: response.ok,
            text: response.statusText
        };

        if (!response.ok) {
            throw new Error(`HTTP error! status: ${response.status}`);
        }

        const data = await response.json();
        return { data, status };
    } catch (error) {
        return {
            data: { "Message": `Error: ${error.message}` },
            status: {
                code: 400,
                ok: false,
                text: "Bad Request"
            }
        };
    }
}

queryResult = {
    if (knightsQuery) {
        const validation = await validateEndpoint(knightsQuery, "search", "q=knight&departmentId=17");
        if (validation) {
            return {
                data: { "Message": validation },
                status: {
                    code: 400,
                    ok: false,
                    text: "Bad Request"
                }
            };
        }
        const result = await fetchQuery(knightsQuery);
        return result;
    } else {
        return {
            data: { "Message": "You need to enter a valid endpoint." },
            status: {
                code: 400,
                ok: false,
                text: "Bad Request"
            }
        };
    }
}

prettyQueryResult = {
    let jsonString;
    if (typeof queryResult === 'string') {
        jsonString = JSON.stringify(JSON.parse(queryResult), null, 2);
    } else {
        jsonString = JSON.stringify(queryResult, null, 2);
    }
    return jsonString;
}

viewof prettyQueryResultContainer = {
    let content;
    if (queryResult.data.Message) {
        content = html`<div class="alert alert-warning m-0">${queryResult.data.Message}</div>`;
    } else {
        content = html`<pre class="card-body m-0" style="background-color: #f8f9fa; max-height: 400px; overflow-y: auto;">${prettyQueryResult}</pre>`;
    }
    
    const badgeClass = queryResult.status.ok ? "bg-success" : "bg-danger";
    
    const container = html`<div class="card">
        <div class="card-header d-flex justify-content-between align-items-center">
            <span>Query result</span>
            <span class="badge ${badgeClass}">${queryResult.status.code} ${queryResult.status.text}</span>
        </div>
        ${content}
    </div>`;
    return container;
}

Having the query result, the only thing left to do is to get the specific information we need. In this case, you can see that the answer is in the same response under the key total. Just to validate that we have the same answer, write in the next cell the total number of objects that contain the term ‘knight’ in the department of Medieval Art.

Code 
viewof knightsTotal = Inputs.number({
    label: "Total number of knights",
    placeholder: "",
    value: 0,
    attributes: {
        class: "form-control mb-3"
    }
});

viewof validationResult = {
    const container = html`<div></div>`;
    
    if (!queryResult.status.ok || queryResult.data.Message) {
        if (knightsTotal !== 0) {  // Only show warning if user has entered a number
            container.innerHTML = `
                <div class="alert alert-warning">
                    Please enter a valid query first before submitting your answer!
                </div>`;
        }
    } else if (queryResult.data.total !== undefined) {
        if (knightsTotal === queryResult.data.total) {
            container.innerHTML = `
                <div class="alert alert-success">
                    Correct! There are ${queryResult.data.total} objects that contain the term 'knight' in the Medieval Art department.
                </div>`;
        } else {
            container.innerHTML = `
                <div class="alert alert-danger">
                    That's not correct. Try again!
                </div>`;
        }
    }
    
    return container;
}

Great! Now you know the basics stepts to get data from an API. In the next chapter, we will show you how to get data that can be tabulated and analyzed.