Skip to content

Commit

Permalink
Add example notebook for error handling (#196)
Browse files Browse the repository at this point in the history 
* Add error handling example

* Fix notebook formatting
  • Loading branch information
@vishal-dharm
vishal-dharm committed Jun 10, 2024
1 parent 063de56 commit d21f205
Showing 1 changed file with 312 additions and 0 deletions.
312 changes: 312 additions & 0 deletions quickstarts/Error_handling.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,312 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "8968a502d25e"
},
"source": [
"##### Copyright 2024 Google LLC."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "a73f56372655"
},
"outputs": [],
"source": [
"# @title Licensed under the Apache License, Version 2.0 (the \\\"License\\\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \\\"AS IS\\\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "35219116d3b3"
},
"source": [
"# Gemini API: Error handling"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "571fb2e6d4ba"
},
"source": [
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Error_handling.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "b58341384860"
},
"source": [
"This Colab notebook demonstrates strategies for handling common errors you might encounter when working with the Gemini API:\n",
"\n",
"* **Transient Errors:** Temporary failures due to network issues, server overload, etc.\n",
"* **Rate Limits:** Restrictions on the number of requests you can make within a certain timeframe.\n",
"* **Timeouts:** When an API call takes too long to complete.\n",
"\n",
"You have two main approaches to explore:\n",
"\n",
"1. **Automatic retries:** A simple way to retry requests when they fail due to transient errors.\n",
"2. **Manual backoff and retry:** A more customizable approach that provides finer control over retry behavior.\n",
"\n",
"\n",
"**Gemini Rate Limits**\n",
"\n",
"The default rate limits for different Gemini models are outlined in the [Gemini API model documentation](https://ai.google.dev/gemini-api/docs/models/gemini#model-variations). If your application requires a higher quota, consider [requesting a rate limit increase](https://ai.google.dev/gemini-api/docs/quota)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "b2bb83e651f4"
},
"outputs": [],
"source": [
"!pip install -U google-generativeai"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "34bf10124280"
},
"outputs": [],
"source": [
"import google.generativeai as genai"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cf16b627705c"
},
"source": [
"### Setup your API key\n",
"\n",
"To run the following cells, store your API key in a Colab Secret named `GOOGLE_API_KEY`. If you don't have an API key or need help creating a Colab Secret, see the [Authentication](https://github.com/google-gemini/cookbook/blob/main/quickstarts/Authentication.ipynb) guide."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "34e7b42a93e3"
},
"outputs": [],
"source": [
"from google.colab import userdata\n",
"\n",
"GOOGLE_API_KEY = userdata.get(\"GOOGLE_API_KEY\")\n",
"genai.configure(api_key=GOOGLE_API_KEY)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "89d975abf7a2"
},
"source": [
"### Automatic retries\n",
"\n",
"The Gemini API's client library offers built-in retry mechanisms for handling transient errors. You can enable this feature by using the `request_options` argument with API calls like `generate_content`, `generate_answer`, `embed_content`, and `generate_content_async`.\n",
"\n",
"**Advantages:**\n",
"\n",
"* **Simplicity:** Requires minimal code changes for significant reliability gains.\n",
"* **Robust:** Effectively addresses most transient errors without additional logic.\n",
"\n",
"**Customize retry behavior:**\n",
"\n",
"Use these settings in [`retry`](https://googleapis.dev/python/google-api-core/latest/retry.html) to customize retry behavior:\n",
"\n",
"* `predicate`: (callable) Determines if an exception is retryable. Default: [`if_transient_error`](https://github.com/googleapis/python-api-core/blob/main/google/api_core/retry/retry_base.py#L75C4-L75C13)\n",
"* `initial`: (float) Initial delay in seconds before the first retry. Default: `1.0`\n",
"* `maximum`: (float) Maximum delay in seconds between retries. Default: `60.0`\n",
"* `multiplier`: (float) Factor by which the delay increases after each retry. Default: `2.0`\n",
"* `timeout`: (float) Total retry duration in seconds. Default: `120.0`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "09d14986b9cf"
},
"outputs": [],
"source": [
"from google.api_core import retry\n",
"\n",
"model = genai.GenerativeModel(\"gemini-1.5-flash-latest\")\n",
"prompt = \"Write a story about a magic backpack.\"\n",
"\n",
"model.generate_content(\n",
" prompt, request_options={\"retry\": retry.Retry(predicate=retry.if_transient_error)}\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1abafce2315c"
},
"source": [
"### Manually increase timeout when responses take time\n",
"\n",
"If you encounter `ReadTimeout` or `DeadlineExceeded` errors, meaning an API call exceeds the default timeout (600 seconds), you can manually adjust it by defining `timeout` in the `request_options` argument."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5bdc7450fcd8"
},
"outputs": [],
"source": [
"model = genai.GenerativeModel(\"gemini-1.5-flash-latest\")\n",
"prompt = \"Write a story about a magic backpack.\"\n",
"\n",
"model.generate_content(\n",
" prompt, request_options={\"timeout\": 900}\n",
") # Increase timeout to 15 minutes"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c27ccc71fa27"
},
"source": [
"**Caution:** While increasing timeouts can be helpful, be mindful of setting them too high, as this can delay error detection and potentially waste resources."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "69dd9fd7f359"
},
"source": [
"### Manually implement backoff and retry with error handling\n",
"\n",
"For finer control over retry behavior and error handling, you can use the [`retry`](https://googleapis.dev/python/google-api-core/latest/retry.html) library (or similar libraries like [`backoff`](https://pypi.org/project/backoff/) and [`tenacity`](https://tenacity.readthedocs.io/en/latest/)). This gives you precise control over retry strategies and allows you to handle specific types of errors differently."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "c134fc5aaecc"
},
"outputs": [],
"source": [
"from google.api_core import retry, exceptions\n",
"\n",
"model = genai.GenerativeModel(\"gemini-1.5-flash-latest\")\n",
"\n",
"\n",
"@retry.Retry(\n",
" predicate=retry.if_transient_error,\n",
" initial=2.0,\n",
" maximum=64.0,\n",
" multiplier=2.0,\n",
" timeout=600,\n",
")\n",
"def generate_with_retry(model, prompt):\n",
" response = model.generate_content(prompt)\n",
" return response\n",
"\n",
"\n",
"prompt = \"Write a one-liner advertisement for magic backpack.\"\n",
"\n",
"generate_with_retry(model=model, prompt=prompt)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "85fc3d0bae17"
},
"source": [
"### Test the error handling with retry mechanism\n",
"\n",
"To validate that your error handling and retry mechanism work as intended, define a `generate_content` function that deliberately raises a `ServiceUnavailable` error on the first call. This setup will help you ensure that the retry decorator successfully handles the transient error and retries the operation."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "981415e25158"
},
"outputs": [],
"source": [
"from google.api_core import retry, exceptions\n",
"\n",
"\n",
"@retry.Retry(\n",
" predicate=retry.if_transient_error,\n",
" initial=2.0,\n",
" maximum=64.0,\n",
" multiplier=2.0,\n",
" timeout=600,\n",
")\n",
"def generate_content_first_fail(model, prompt):\n",
" if not hasattr(generate_content_first_fail, \"call_counter\"):\n",
" generate_content_first_fail.call_counter = 0\n",
"\n",
" generate_content_first_fail.call_counter += 1\n",
"\n",
" try:\n",
" if generate_content_first_fail.call_counter == 1:\n",
" raise exceptions.ServiceUnavailable(\"Service Unavailable\")\n",
"\n",
" response = model.generate_content(prompt)\n",
" return response.text\n",
" except exceptions.ServiceUnavailable as e:\n",
" print(f\"Error: {e}\")\n",
" raise\n",
"\n",
"\n",
"model = genai.GenerativeModel(\"gemini-1.5-flash-latest\")\n",
"prompt = \"Write a one-liner advertisement for magic backpack.\"\n",
"\n",
"generate_content_first_fail(model=model, prompt=prompt)"
]
}
],
"metadata": {
"colab": {
"name": "Error_handling.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

0 comments on commit d21f205

Please sign in to comment.