{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d3f51c7",
   "metadata": {
    "tags": [
     "hide-output"
    ]
   },
   "outputs": [],
   "source": [
    "!pip install pandas requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebba03b7-34c4-4b49-94d8-112c7176ac7f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import requests\n",
    "API_URL = \"https://mastapp.site/json\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76fa365c-b6d9-479f-99d8-d0d53ec30e8e",
   "metadata": {},
   "source": [
    "# REST API Examples\n",
    "\n",
    "In this notebook we explore searching for metadata from the REST API. The REST API provides a method to programmatically extract a JSON representation of the meta data from the API."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba8f861b-b74f-40d1-9df2-6b4f5b8f6edd",
   "metadata": {},
   "source": [
    "## Querying Shots with the REST API\n",
    "\n",
    "We're going to use the python `requests` library to query metadata from the database. All we need to do to get a result is to query the database with a HTTP GET at the appropriate endpoint. For example, to get information about different experimental shots we can query the `/json/shots/` endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a722af66-9731-4878-b2df-9c9ba7ac0be7",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(f'{API_URL}/shots')\n",
    "result = response.json()\n",
    "print(f\"Query returned status code: {response.status_code}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "473e3069",
   "metadata": {},
   "source": [
    "The shots endpoint returns a JSON payload with a list of shots. Let's look at the first element from the payload:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2611f771",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(result['items'][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7a8cfac",
   "metadata": {},
   "source": [
    "Each item in the list is a json object. This contains the meta-data items that corresponded to our query. In this case, each item contains information about a different MAST shot. Each item has lots of information about different shots, for example the shot ID, the campaign the shot was part of, the pre- and post-shot description by investigators.\n",
    "\n",
    "For more information on the what's returned by the API you can look at the endpoint documentation:\n",
    "\n",
    "https://mastapp.site/redoc\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f19e602d",
   "metadata": {},
   "source": [
    "Of course, we can read all this JSON data directly into common python data analysis packages, for example, we can create a `pandas` dataframe directly from the endpoint data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66c264c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_json(f'{API_URL}/shots', lines=True)['items'][0]\n",
    "df = pd.DataFrame(df)\n",
    "df.head()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9de70947",
   "metadata": {},
   "source": [
    "### Searching & Filtering Data\n",
    "\n",
    "All REST API endpoints can take query parameters to filter the data returned. For example, we can return all shots for the `M9` campaign by using the approrpiate query string.\n",
    "\n",
    "We can query for everything from the `M9` campaign by adding `?filters=campaign$eq:M9` to our query string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed99237a",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_json(f'{API_URL}/shots?filters=campaign$eq:M9', lines=True)['items'][0]\n",
    "df = pd.DataFrame(df)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2e54b05-98f5-4ff2-a2e9-be3e356e0c3a",
   "metadata": {},
   "source": [
    "### Pagination \n",
    "The REST API responses are _paginated_, meaning that only a subset of the full items are returned with each query. Pagination is used to limit the total number of requests made by each user to prevent any single user overloading the server with huge data requests.\n",
    "\n",
    "Pagination information is included in the response and corresponds to [RFC 8288](https://datatracker.ietf.org/doc/html/rfc8288). The response contains cursors at the bottom of the response:\n",
    "\n",
    " - `current_page` - this is the cursor relating to the current page of results.\n",
    " - `next_page` - this is the cursor for the next page of results.\n",
    " - `previous_page` - this is the cursor for the previous page of results, will result in `null` if on the first page.\n",
    "\n",
    "You can control the pagination using the following options as query arguments:\n",
    " - `cursor=xx` to set the cursor for the displayed set of results\n",
    " - `size=xx` to set the number of results returned by each page."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80d840d5-ccf4-4ed2-9fe4-33798f2ae62f",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(f'{API_URL}/shots?size=2&cursor=Pmk6MzAxMTE%3D')\n",
    "result = response.json()\n",
    "headers = response.headers\n",
    "\n",
    "print(\"Current page cursor\", result['current_page'])\n",
    "print(\"Next page cursor\", result['next_page'])\n",
    "print(\"Previous page cursor\", result['previous_page'])\n",
    "\n",
    "df = pd.DataFrame(result['items'])\n",
    "df"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}