Vision API 可以使用对象本地化功能检测并提取图片中的多个对象。
对象本地化功能可识别图片中的多个对象,并为图片中的每个对象提供一个 LocalizedObjectAnnotation。每个 LocalizedObjectAnnotation 标识了对象相关信息、对象位置以及对象所属图片区域的矩形边界。
对象本地化功能可识别图片中显眼和不太显眼的对象。
对象信息仅以英文形式返回。Cloud Translation 可以将英文标签翻译成各种其他语言。
例如,API 会返回上图中对象的以下信息和边界位置数据:
| 名称 | mid | 得分 | 边界 |
|---|---|---|---|
| Bicycle wheel | /m/01bqk0 | 0.89648587 | (0.32076266, 0.78941387)、(0.43812272, 0.78941387)、(0.43812272, 0.97331065)、(0.32076266, 0.97331065) |
| 骑车 | /m/0199g | 0.886761 | (0.312, 0.6616471)、(0.638353, 0.6616471)、(0.638353, 0.9705882)、(0.312, 0.9705882) |
| Bicycle wheel | /m/01bqk0 | 0.6345275 | (0.5125398, 0.760708)、(0.6256646, 0.760708)、(0.6256646, 0.94601655)、(0.5125398, 0.94601655) |
| Picture frame | /m/06z37_ | 0.6207608 | (0.79177403, 0.16160682)、(0.97047985, 0.16160682)、(0.97047985, 0.31348917)、(0.79177403, 0.31348917) |
| Tire | /m/0h9mv | 0.55886006 | (0.32076266, 0.78941387)、(0.43812272, 0.78941387)、(0.43812272, 0.97331065)、(0.32076266, 0.97331065) |
| Door | /m/02dgv | 0.5160098 | (0.77569866, 0.37104446)、(0.9412425, 0.37104446)、(0.9412425, 0.81507325)、(0.77569866, 0.81507325) |
mid 包含与标签的 Google 知识图谱条目相对应并由机器生成的标识符 (MID)。如需了解如何检查 mid 值,请参阅 Google Knowledge Graph Search API 文档。
对象本地化请求
设置您的 Google Cloud 项目和身份验证
如果您尚未创建 Google Cloud 项目,请立即创建。展开本部分可查看相关说明。
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
-
Create a project: To create a project, you need the Project Creator role
(
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission. Learn how to grant roles.
-
Verify that billing is enabled for your Google Cloud project.
-
Enable the Vision API.
Roles required to enable APIs
To enable APIs, you need the Service Usage Admin IAM role (
roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enablepermission. Learn how to grant roles. -
Install the Google Cloud CLI.
-
配置 gcloud CLI 以使用您的联合身份。
如需了解详情,请参阅使用联合身份登录 gcloud CLI。
-
如需初始化 gcloud CLI,请运行以下命令:
gcloud init检测本地图片中的对象
您可以使用 Vision API 对本地图片文件执行特征检测。
对于 REST 请求,请将图片文件的内容作为 base64 编码的字符串在请求正文中发送。
对于
gcloud和客户端库请求,请在请求中指定本地图片的路径。REST
在使用任何请求数据之前,请先进行以下替换:
- BASE64_ENCODED_IMAGE:二进制图片数据的 base64 表示(ASCII 字符串)。此字符串应类似于以下字符串:
/9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==
- RESULTS_INT:(可选)要返回的结果的整数值。如果您省略
"maxResults"字段及其值,则 API 会默认返回 10 个结果。此字段不适用于以下功能类型:TEXT_DETECTION、DOCUMENT_TEXT_DETECTION或CROP_HINTS。 - PROJECT_ID:您的 Google Cloud 项目 ID。
HTTP 方法和网址:
POST https://vision.googleapis.com/v1/images:annotate
请求 JSON 正文:
{ "requests": [ { "image": { "content": "BASE64_ENCODED_IMAGE" }, "features": [ { "maxResults": RESULTS_INT, "type": "OBJECT_LOCALIZATION" }, ] } ] }如需发送请求,请选择以下方式之一:
curl
将请求正文保存在名为
request.json的文件中,然后执行以下命令:curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "x-goog-user-project: PROJECT_ID" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://vision.googleapis.com/v1/images:annotate"PowerShell
将请求正文保存在名为
request.json的文件中,然后执行以下命令:$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "PROJECT_ID" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content如果请求成功,服务器将返回一个
200 OKHTTP 状态代码以及 JSON 格式的响应。响应:
响应
{ "responses": [ { "localizedObjectAnnotations": [ { "mid": "/m/01bqk0", "name": "Bicycle wheel", "score": 0.89648587, "boundingPoly": { "normalizedVertices": [ { "x": 0.32076266, "y": 0.78941387 }, { "x": 0.43812272, "y": 0.78941387 }, { "x": 0.43812272, "y": 0.97331065 }, { "x": 0.32076266, "y": 0.97331065 } ] } }, { "mid": "/m/0199g", "name": "Bicycle", "score": 0.886761, "boundingPoly": { "normalizedVertices": [ { "x": 0.312, "y": 0.6616471 }, { "x": 0.638353, "y": 0.6616471 }, { "x": 0.638353, "y": 0.9705882 }, { "x": 0.312, "y": 0.9705882 } ] } }, { "mid": "/m/01bqk0", "name": "Bicycle wheel", "score": 0.6345275, "boundingPoly": { "normalizedVertices": [ { "x": 0.5125398, "y": 0.760708 }, { "x": 0.6256646, "y": 0.760708 }, { "x": 0.6256646, "y": 0.94601655 }, { "x": 0.5125398, "y": 0.94601655 } ] } }, { "mid": "/m/06z37_", "name": "Picture frame", "score": 0.6207608, "boundingPoly": { "normalizedVertices": [ { "x": 0.79177403, "y": 0.16160682 }, { "x": 0.97047985, "y": 0.16160682 }, { "x": 0.97047985, "y": 0.31348917 }, { "x": 0.79177403, "y": 0.31348917 } ] } }, { "mid": "/m/0h9mv", "name": "Tire", "score": 0.55886006, "boundingPoly": { "normalizedVertices": [ { "x": 0.32076266, "y": 0.78941387 }, { "x": 0.43812272, "y": 0.78941387 }, { "x": 0.43812272, "y": 0.97331065 }, { "x": 0.32076266, "y": 0.97331065 } ] } }, { "mid": "/m/02dgv", "name": "Door", "score": 0.5160098, "boundingPoly": { "normalizedVertices": [ { "x": 0.77569866, "y": 0.37104446 }, { "x": 0.9412425, "y": 0.37104446 }, { "x": 0.9412425, "y": 0.81507325 }, { "x": 0.77569866, "y": 0.81507325 } ] } } ] } ] }Go
试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Go 设置说明进行操作。 如需了解详情,请参阅 Vision Go API 参考文档。
如需向 Vision 进行身份验证,请设置应用默认凭证。如需了解详情,请参阅为本地开发环境设置身份验证。
// localizeObjects gets objects and bounding boxes from the Vision API for an image at the given file path. func localizeObjects(w io.Writer, file string) error { ctx := context.Background() client, err := vision.NewImageAnnotatorClient(ctx) if err != nil { return err } f, err := os.Open(file) if err != nil { return err } defer f.Close() image, err := vision.NewImageFromReader(f) if err != nil { return err } annotations, err := client.LocalizeObjects(ctx, image, nil) if err != nil { return err } if len(annotations) == 0 { fmt.Fprintln(w, "No objects found.") return nil } fmt.Fprintln(w, "Objects:") for _, annotation := range annotations { fmt.Fprintln(w, annotation.Name) fmt.Fprintln(w, annotation.Score) for _, v := range annotation.BoundingPoly.NormalizedVertices { fmt.Fprintf(w, "(%f,%f)\n", v.X, v.Y) } } return nil }Java
在试用此示例之前,请按照Vision API 快速入门:使用客户端库中的 Java 设置说明进行操作。如需了解详情,请参阅 Vision API Java 参考文档。
/** * Detects localized objects in the specified local image. * * @param filePath The path to the file to perform localized object detection on. * @throws Exception on errors while closing the client. * @throws IOException on Input/Output errors. */ public static void detectLocalizedObjects(String filePath) throws IOException { List<AnnotateImageRequest> requests = new ArrayList<>(); ByteString imgBytes = ByteString.readFrom(new FileInputStream(filePath)); Image img = Image.newBuilder().setContent(imgBytes).build(); AnnotateImageRequest request = AnnotateImageRequest.newBuilder() .addFeatures(Feature.newBuilder().setType(Type.OBJECT_LOCALIZATION)) .setImage(img) .build(); requests.add(request); // Initialize client that will be used to send requests. This client only needs to be created // once, and can be reused for multiple requests. After completing all of your requests, call // the "close" method on the client to safely clean up any remaining background resources. try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) { // Perform the request BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests); List<AnnotateImageResponse> responses = response.getResponsesList(); // Display the results for (AnnotateImageResponse res : responses) { for (LocalizedObjectAnnotation entity : res.getLocalizedObjectAnnotationsList()) { System.out.format("Object name: %s%n", entity.getName()); System.out.format("Confidence: %s%n", entity.getScore()); System.out.format("Normalized Vertices:%n"); entity .getBoundingPoly() .getNormalizedVerticesList() .forEach(vertex -> System.out.format("- (%s, %s)%n", vertex.getX(), vertex.getY())); } } } }Node.js
试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Node.js 设置说明进行操作。 如需了解详情,请参阅 Vision Node.js API 参考文档。
如需向 Vision 进行身份验证,请设置应用默认凭证。如需了解详情,请参阅为本地开发环境设置身份验证。
// Imports the Google Cloud client libraries const vision = require('@google-cloud/vision'); const fs = require('fs'); // Creates a client const client = new vision.ImageAnnotatorClient(); /** * TODO(developer): Uncomment the following line before running the sample. */ // const fileName = `/path/to/localImage.png`; const request = { image: {content: fs.readFileSync(fileName)}, }; const [result] = await client.objectLocalization(request); const objects = result.localizedObjectAnnotations; objects.forEach(object => { console.log(`Name: ${object.name}`); console.log(`Confidence: ${object.score}`); const vertices = object.boundingPoly.normalizedVertices; vertices.forEach(v => console.log(`x: ${v.x}, y:${v.y}`)); });Python
试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Python 设置说明进行操作。 如需了解详情,请参阅 Vision Python API 参考文档。
如需向 Vision 进行身份验证,请设置应用默认凭证。如需了解详情,请参阅为本地开发环境设置身份验证。
def localize_objects(path): """Localize objects in the local image. Args: path: The path to the local file. """ from google.cloud import vision client = vision.ImageAnnotatorClient() with open(path, "rb") as image_file: content = image_file.read() image = vision.Image(content=content) objects = client.object_localization(image=image).localized_object_annotations print(f"Number of objects found: {len(objects)}") for object_ in objects: print(f"\n{object_.name} (confidence: {object_.score})") print("Normalized bounding polygon vertices: ") for vertex in object_.bounding_poly.normalized_vertices: print(f" - ({vertex.x}, {vertex.y})")其他语言
C#: 请按照客户端库页面上的 C# 设置说明操作,然后访问 .NET 版 Vision 参考文档。
PHP: 请按照客户端库页面上的 PHP 设置说明操作,然后访问 PHP 版 Vision 参考文档。
Ruby 版: 请按照客户端库页面上的 Ruby 设置说明操作,然后访问 Ruby 版 Vision 参考文档。
检测远程图片中的对象
您可以使用 Vision API 对位于 Cloud Storage 或网络中的远程图片文件执行特征检测。如需发送远程文件请求,请在请求正文中指定文件的网址或 Cloud Storage URI。
REST
在使用任何请求数据之前,请先进行以下替换:
- CLOUD_STORAGE_IMAGE_URI:Cloud Storage 存储桶中有效图片文件的路径。您必须至少拥有该文件的读取权限。
示例:
https://cloud.google.com/vision/docs/images/bicycle_example.png
- RESULTS_INT:(可选)要返回的结果的整数值。如果您省略
"maxResults"字段及其值,则 API 会默认返回 10 个结果。此字段不适用于以下功能类型:TEXT_DETECTION、DOCUMENT_TEXT_DETECTION或CROP_HINTS。 - PROJECT_ID:您的 Google Cloud 项目 ID。
HTTP 方法和网址:
POST https://vision.googleapis.com/v1/images:annotate
请求 JSON 正文:
{ "requests": [ { "image": { "source": { "imageUri": "CLOUD_STORAGE_IMAGE_URI" } }, "features": [ { "maxResults": RESULTS_INT, "type": "OBJECT_LOCALIZATION" }, ] } ] }如需发送请求,请选择以下方式之一:
curl
将请求正文保存在名为
request.json的文件中,然后执行以下命令:curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "x-goog-user-project: PROJECT_ID" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://vision.googleapis.com/v1/images:annotate"PowerShell
将请求正文保存在名为
request.json的文件中,然后执行以下命令:$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "PROJECT_ID" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content如果请求成功,服务器将返回一个
200 OKHTTP 状态代码以及 JSON 格式的响应。响应:
响应
{ "responses": [ { "localizedObjectAnnotations": [ { "mid": "/m/01bqk0", "name": "Bicycle wheel", "score": 0.89648587, "boundingPoly": { "normalizedVertices": [ { "x": 0.32076266, "y": 0.78941387 }, { "x": 0.43812272, "y": 0.78941387 }, { "x": 0.43812272, "y": 0.97331065 }, { "x": 0.32076266, "y": 0.97331065 } ] } }, { "mid": "/m/0199g", "name": "Bicycle", "score": 0.886761, "boundingPoly": { "normalizedVertices": [ { "x": 0.312, "y": 0.6616471 }, { "x": 0.638353, "y": 0.6616471 }, { "x": 0.638353, "y": 0.9705882 }, { "x": 0.312, "y": 0.9705882 } ] } }, { "mid": "/m/01bqk0", "name": "Bicycle wheel", "score": 0.6345275, "boundingPoly": { "normalizedVertices": [ { "x": 0.5125398, "y": 0.760708 }, { "x": 0.6256646, "y": 0.760708 }, { "x": 0.6256646, "y": 0.94601655 }, { "x": 0.5125398, "y": 0.94601655 } ] } }, { "mid": "/m/06z37_", "name": "Picture frame", "score": 0.6207608, "boundingPoly": { "normalizedVertices": [ { "x": 0.79177403, "y": 0.16160682 }, { "x": 0.97047985, "y": 0.16160682 }, { "x": 0.97047985, "y": 0.31348917 }, { "x": 0.79177403, "y": 0.31348917 } ] } }, { "mid": "/m/0h9mv", "name": "Tire", "score": 0.55886006, "boundingPoly": { "normalizedVertices": [ { "x": 0.32076266, "y": 0.78941387 }, { "x": 0.43812272, "y": 0.78941387 }, { "x": 0.43812272, "y": 0.97331065 }, { "x": 0.32076266, "y": 0.97331065 } ] } }, { "mid": "/m/02dgv", "name": "Door", "score": 0.5160098, "boundingPoly": { "normalizedVertices": [ { "x": 0.77569866, "y": 0.37104446 }, { "x": 0.9412425, "y": 0.37104446 }, { "x": 0.9412425, "y": 0.81507325 }, { "x": 0.77569866, "y": 0.81507325 } ] } } ] } ] }Go
试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Go 设置说明进行操作。 如需了解详情,请参阅 Vision Go API 参考文档。
如需向 Vision 进行身份验证,请设置应用默认凭证。如需了解详情,请参阅为本地开发环境设置身份验证。
// localizeObjects gets objects and bounding boxes from the Vision API for an image at the given file path. func localizeObjectsURI(w io.Writer, file string) error { ctx := context.Background() client, err := vision.NewImageAnnotatorClient(ctx) if err != nil { return err } image := vision.NewImageFromURI(file) annotations, err := client.LocalizeObjects(ctx, image, nil) if err != nil { return err } if len(annotations) == 0 { fmt.Fprintln(w, "No objects found.") return nil } fmt.Fprintln(w, "Objects:") for _, annotation := range annotations { fmt.Fprintln(w, annotation.Name) fmt.Fprintln(w, annotation.Score) for _, v := range annotation.BoundingPoly.NormalizedVertices { fmt.Fprintf(w, "(%f,%f)\n", v.X, v.Y) } } return nil }Java
在试用此示例之前,请按照Vision API 快速入门:使用客户端库中的 Java 设置说明进行操作。如需了解详情,请参阅 Vision API Java 参考文档。
/** * Detects localized objects in a remote image on Google Cloud Storage. * * @param gcsPath The path to the remote file on Google Cloud Storage to detect localized objects * on. * @throws Exception on errors while closing the client. * @throws IOException on Input/Output errors. */ public static void detectLocalizedObjectsGcs(String gcsPath) throws IOException { List<AnnotateImageRequest> requests = new ArrayList<>(); ImageSource imgSource = ImageSource.newBuilder().setGcsImageUri(gcsPath).build(); Image img = Image.newBuilder().setSource(imgSource).build(); AnnotateImageRequest request = AnnotateImageRequest.newBuilder() .addFeatures(Feature.newBuilder().setType(Type.OBJECT_LOCALIZATION)) .setImage(img) .build(); requests.add(request); // Initialize client that will be used to send requests. This client only needs to be created // once, and can be reused for multiple requests. After completing all of your requests, call // the "close" method on the client to safely clean up any remaining background resources. try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) { // Perform the request BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests); List<AnnotateImageResponse> responses = response.getResponsesList(); client.close(); // Display the results for (AnnotateImageResponse res : responses) { for (LocalizedObjectAnnotation entity : res.getLocalizedObjectAnnotationsList()) { System.out.format("Object name: %s%n", entity.getName()); System.out.format("Confidence: %s%n", entity.getScore()); System.out.format("Normalized Vertices:%n"); entity .getBoundingPoly() .getNormalizedVerticesList() .forEach(vertex -> System.out.format("- (%s, %s)%n", vertex.getX(), vertex.getY())); } } } }Node.js
试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Node.js 设置说明进行操作。 如需了解详情,请参阅 Vision Node.js API 参考文档。
如需向 Vision 进行身份验证,请设置应用默认凭证。如需了解详情,请参阅为本地开发环境设置身份验证。
// Imports the Google Cloud client libraries const vision = require('@google-cloud/vision'); // Creates a client const client = new vision.ImageAnnotatorClient(); /** * TODO(developer): Uncomment the following line before running the sample. */ // const gcsUri = `gs://bucket/bucketImage.png`; const [result] = await client.objectLocalization(gcsUri); const objects = result.localizedObjectAnnotations; objects.forEach(object => { console.log(`Name: ${object.name}`); console.log(`Confidence: ${object.score}`); const veritices = object.boundingPoly.normalizedVertices; veritices.forEach(v => console.log(`x: ${v.x}, y:${v.y}`)); });Python
试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Python 设置说明进行操作。 如需了解详情,请参阅 Vision Python API 参考文档。
如需向 Vision 进行身份验证,请设置应用默认凭证。如需了解详情,请参阅为本地开发环境设置身份验证。
def localize_objects_uri(uri): """Localize objects in the image on Google Cloud Storage Args: uri: The path to the file in Google Cloud Storage (gs://...) """ from google.cloud import vision client = vision.ImageAnnotatorClient() image = vision.Image() image.source.image_uri = uri objects = client.object_localization(image=image).localized_object_annotations print(f"Number of objects found: {len(objects)}") for object_ in objects: print(f"\n{object_.name} (confidence: {object_.score})") print("Normalized bounding polygon vertices: ") for vertex in object_.bounding_poly.normalized_vertices: print(f" - ({vertex.x}, {vertex.y})")gcloud
如需检测图片中的标签,请使用
gcloud ml vision detect-objects命令,如以下示例所示:gcloud ml vision detect-objects https://cloud.google.com/vision/docs/images/bicycle_example.png
其他语言
C#: 请按照客户端库页面上的 C# 设置说明操作,然后访问 .NET 版 Vision 参考文档。
PHP: 请按照客户端库页面上的 PHP 设置说明操作,然后访问 PHP 版 Vision 参考文档。
Ruby 版: 请按照客户端库页面上的 Ruby 设置说明操作,然后访问 Ruby 版 Vision 参考文档。
试用
请尝试使用以下工具执行对象检测和本地化。您可以使用已指定的图片 (
https://cloud.google.com/vision/docs/images/bicycle_example.png) 或指定您自己的图片。选择执行即可发送请求。
图片来源:Unsplash 用户 Bogdan Dada。 请求正文:
{ "requests": [ { "features": [ { "maxResults": 10, "type": "OBJECT_LOCALIZATION" } ], "image": { "source": { "imageUri": "https://cloud.google.com/vision/docs/images/bicycle_example.png" } } } ] } - BASE64_ENCODED_IMAGE:二进制图片数据的 base64 表示(ASCII 字符串)。此字符串应类似于以下字符串: