{ "cells": [ { "cell_type": "markdown", "id": "a9e360da", "metadata": {}, "source": [ "# Working with a sequence of images\n", "In this notebook, we will look at how to work with a sequence of images, and possibly change some parameters on the fly. As in all examples, we will start with setting the Keras backend." ] }, { "cell_type": "markdown", "id": "b303c558", "metadata": {}, "source": [ "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tue-bmd/zea/blob/main/docs/source/notebooks/pipeline/zea_sequence_example.ipynb)\n", " \n", "[![View on GitHub](https://img.shields.io/badge/GitHub-View%20Source-blue?logo=github)](https://github.com/tue-bmd/zea/blob/main/docs/source/notebooks/pipeline/zea_sequence_example.ipynb)\n", " \n", "[![Hugging Face dataset](https://img.shields.io/badge/Hugging%20Face-Dataset-yellow?logo=huggingface)](https://huggingface.co/datasets/zeahub/zea-carotid-2023)" ] }, { "cell_type": "code", "execution_count": 1, "id": "6d6dd155", "metadata": {}, "outputs": [], "source": [ "%%capture\n", "%pip install zea" ] }, { "cell_type": "code", "execution_count": 2, "id": "b41342fd", "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, { "cell_type": "code", "execution_count": 3, "id": "7e4af10b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m\u001b[38;5;36mzea\u001b[0m\u001b[0m: Using backend 'jax'\n" ] } ], "source": [ "import keras\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from matplotlib import animation\n", "from IPython.display import HTML\n", "\n", "import zea\n", "from zea import init_device, load_file\n", "from zea.visualize import set_mpl_style" ] }, { "cell_type": "code", "execution_count": 4, "id": "2047ca56", "metadata": { "tags": [ "parameters" ] }, "outputs": [], "source": [ "n_frames = 15\n", "n_tx = 11\n", "n_tx_total = 127" ] }, { "cell_type": "markdown", "id": "6de3d4f2", "metadata": {}, "source": [ "We will work with the GPU if available, and initialize using `init_device` to pick the best available device. Also, (optionally), we will set the matplotlib style for plotting." ] }, { "cell_type": "code", "execution_count": 5, "id": "e55d6107", "metadata": {}, "outputs": [], "source": [ "init_device(verbose=False)\n", "set_mpl_style()" ] }, { "cell_type": "markdown", "id": "7db13230", "metadata": {}, "source": [ "We create a small helper function to animate the sequence of images." ] }, { "cell_type": "code", "execution_count": 6, "id": "138c62e4", "metadata": {}, "outputs": [], "source": [ "def animate_images(images, scan, interval=100, cmap=\"gray\"):\n", " \"\"\"Helper function to animate a list of images.\"\"\"\n", " fig, ax = plt.subplots(figsize=(5, 4), dpi=80)\n", " xlims_mm = [v * 1e3 for v in scan.xlims]\n", " zlims_mm = [v * 1e3 for v in scan.zlims]\n", " im = ax.imshow(\n", " np.array(images[0]),\n", " animated=True,\n", " cmap=cmap,\n", " extent=[xlims_mm[0], xlims_mm[1], zlims_mm[1], zlims_mm[0]],\n", " )\n", " ax.set_xlabel(\"X (mm)\")\n", " ax.set_ylabel(\"Z (mm)\")\n", "\n", " def update(frame):\n", " im.set_array(np.array(images[frame]))\n", " return [im]\n", "\n", " ani = animation.FuncAnimation(\n", " fig,\n", " update,\n", " frames=len(images),\n", " blit=True,\n", " interval=interval,\n", " )\n", " plt.close(fig)\n", " return HTML(ani.to_jshtml(fps=1000 // interval, embed_frames=True, default_mode=\"reflect\"))" ] }, { "cell_type": "markdown", "id": "449d254c", "metadata": {}, "source": [ "Let's initialize a default B-mode ultrasound image formation pipeline." ] }, { "cell_type": "code", "execution_count": 7, "id": "39a27a22", "metadata": {}, "outputs": [], "source": [ "pipeline = zea.Pipeline.from_default(pfield=False, with_batch_dim=False)" ] }, { "cell_type": "markdown", "id": "c369482f", "metadata": {}, "source": [ "We will load a sequence of acquired RF data frames (carotid scan) and reconstruct a B-mode image from each frame. We will then animate the sequence of images. But first let's load the data and parameters." ] }, { "cell_type": "code", "execution_count": 8, "id": "f79441a3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m\u001b[38;5;36mzea\u001b[0m\u001b[0m: \u001b[38;5;214mWARNING\u001b[0m The probe geometry in the data file does not match the probe geometry of the probe. The probe geometry has been updated to match the data file.\n" ] } ], "source": [ "# this can take a while to download\n", "file_path = \"hf://zeahub/zea-carotid-2023/2_cross_bifur_right_0000.hdf5\" # ~25GB\n", "# so let's use the smaller one by default:\n", "file_path = \"hf://zeahub/zea-carotid-2023/2_cross_bifur_right_0000_small.hdf5\" # ~2.5GB\n", "\n", "frames = list(range(n_frames)) # use first 15 frames for demonstration\n", "data, scan, probe = load_file(file_path, \"raw_data\", indices=frames)\n", "\n", "scan.set_transmits(n_tx) # reduce number of transmits for faster processing\n", "scan.zlims = (0, 0.04) # reduce z-limits a bit for better visualizations\n", "scan.xlims = probe.xlims\n", "scan.n_ch = data.shape[-1] # rf data\n", "\n", "config = zea.Config(dynamic_range=(-40, 0))" ] }, { "cell_type": "markdown", "id": "481edac0", "metadata": {}, "source": [ "## Reconstructing a sequence of B-mode images" ] }, { "cell_type": "code", "execution_count": 9, "id": "eabec7f1", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m\u001b[38;5;36mzea\u001b[0m\u001b[0m: Loading cached result for compute_pfield.\n", "\u001b[1m15/15\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m8s\u001b[0m 375ms/step\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n", " \n", "
\n", " \n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
\n", "
\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "images = []\n", "n_frames = data.shape[0]\n", "progbar = keras.utils.Progbar(n_frames, stateful_metrics=[\"frame\"])\n", "\n", "params = pipeline.prepare_parameters(probe, scan, config)\n", "\n", "for frame_no in range(n_frames):\n", " output = pipeline(data=data[frame_no, scan.selected_transmits], **params)\n", " image = output.pop(\"data\")\n", " params = output\n", " images.append(image)\n", " progbar.update(frame_no + 1)\n", "\n", "animate_images(images, scan, interval=100, cmap=\"gray\")" ] }, { "cell_type": "markdown", "id": "364238c4", "metadata": {}, "source": [ "## Change transmits on the fly" ] }, { "cell_type": "markdown", "id": "9af5147a", "metadata": {}, "source": [ "We now used 11 transmits throughout for every frame. We can also sweep through the the transmits for each frame to see how it affects the image quality." ] }, { "cell_type": "code", "execution_count": 10, "id": "43aa9551", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m15/15\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 58ms/step\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n", " \n", "
\n", " \n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
\n", "
\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "images = []\n", "n_frames = len(frames)\n", "progbar = keras.utils.Progbar(n_frames, stateful_metrics=[\"frame\"])\n", "\n", "for idx, frame_no in enumerate(frames):\n", " tx_idx = int(round(idx * (n_tx_total - 1) / (n_frames - 1)))\n", " scan.set_transmits([tx_idx])\n", " raw_data_frame = data[frame_no, tx_idx][None, ...]\n", " with zea.log.set_level(\"WARNING\"): # to surpress info messages\n", " params = pipeline.prepare_parameters(probe, scan)\n", " output = pipeline(data=raw_data_frame, **params)\n", " images.append(output[\"data\"])\n", " progbar.update(idx + 1)\n", "\n", "animate_images(images, scan, interval=100, cmap=\"gray\")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.11" } }, "nbformat": 4, "nbformat_minor": 5 }