建立大型自訂字典偵測工具

本主題說明如何建立及重建大型自訂字典。並說明幾種錯誤情況。

何時應選擇大型自訂字典，而非一般自訂字典

如果您有數萬個機密字詞或詞組需要掃描內容，可以使用一般自訂字典偵測工具。如果您的字詞數量較多，或字詞清單經常變更，建議您建立大型自訂字典，可支援數千萬個字詞。

大型自訂字典與其他自訂 infoType 的差異

大型自訂字典與其他自訂 infoType 的不同之處，在於每個大型自訂字典都有兩個元件：

  • 建立及定義的詞組清單。這份清單會以文字檔案的形式儲存在 Cloud Storage 中，或以資料欄的形式儲存在 BigQuery 資料表中。
  • Sensitive Data Protection 會產生字典檔案，並儲存在 Cloud Storage。字典檔案由字詞清單副本和布隆篩選器組成，有助於搜尋和比對。

建立大型自訂字典

本節說明如何建立、編輯及重建大型自訂字典。

建立字詞清單

建立清單，列出您希望新 infoType 偵測工具搜尋的所有字詞和詞組。執行下列其中一個步驟：

  • 將文字檔案放入 Cloud Storage bucket，每個字詞各占一行。
  • 將 BigQuery 資料表的一個資料欄指定為字詞和詞組的容器。每個項目在資料欄中都應有專屬的資料列。您可以使用現有的 BigQuery 資料表，只要所有字典字詞和詞組都在同一欄即可。

如果字詞清單過大，Sensitive Data Protection 可能無法處理。如果看到錯誤訊息，請參閱本主題後續的「疑難排解錯誤」一節。

建立儲存的 infoType

建立字詞清單後，請使用 Sensitive Data Protection 建立字典：

控制台

  1. 在 Cloud Storage bucket 中建立新資料夾，Sensitive Data Protection 會將產生的字典儲存在該資料夾。

    Sensitive Data Protection 會在您指定的位置建立資料夾，其中包含字典檔案。

  2. 前往 Google Cloud 控制台的「Create infoType」(建立 infoType) 頁面。

    前往「Create infoType」(建立 infoType)

  3. 在「類型」部分，選取「大型自訂字典」。

  4. 在「InfoType ID」部分，輸入已儲存 infoType 的 ID。

    設定檢查和去識別化工作時，您會用到這個 ID。名稱可使用英文字母、數字、連字號和底線。

  5. 在「InfoType 顯示名稱」部分，輸入已儲存 infoType 的名稱。

    名稱中可以使用空格和標點符號。

  6. 在「Description」(說明) 中，輸入已儲存 infoType 的偵測內容說明。

  7. 在「Storage type」(儲存空間類型) 部分，選取字詞清單的位置：

    • BigQuery：輸入專案 ID、資料集 ID 和資料表 ID。在「Field name」(欄位名稱) 欄位中，輸入資料欄 ID。您最多可以從資料表中指定一個資料欄。
    • Google Cloud Storage：輸入檔案路徑。

  8. 在「Output bucket or folder」(輸出值區或資料夾) 中，輸入您在步驟 1 建立的資料夾 Cloud Storage 位置。

  9. 點選「建立」。

系統會顯示已儲存 infoType 的摘要。字典產生完畢且新的儲存 infoType 可供使用時，infoType 的狀態會顯示為「Ready」(就緒)。

C#

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。


using System;
using Google.Api.Gax.ResourceNames;
using Google.Cloud.Dlp.V2;

public class CreateStoredInfoTypes
{
    public static StoredInfoType Create(
        string projectId,
        string outputPath,
        string storedInfoTypeId)
    {
        // Instantiate the dlp client.
        var dlp = DlpServiceClient.Create();

        // Construct the stored infotype config by specifying the public table and 
        // cloud storage output path.
        var storedInfoTypeConfig = new StoredInfoTypeConfig
        {
            DisplayName = "Github Usernames",
            Description = "Dictionary of Github usernames used in commits.",
            LargeCustomDictionary = new LargeCustomDictionaryConfig
            {
                BigQueryField = new BigQueryField
                {
                    Table = new BigQueryTable
                    {
                        DatasetId = "samples",
                        ProjectId = "bigquery-public-data",
                        TableId = "github_nested"
                    },
                    Field = new FieldId
                    {
                        Name = "actor"
                    }
                },
                OutputPath = new CloudStoragePath
                {
                    Path = outputPath
                }
            },
        };

        // Construct the request.
        var request = new CreateStoredInfoTypeRequest
        {
            ParentAsLocationName = new LocationName(projectId, "global"),
            Config = storedInfoTypeConfig,
            StoredInfoTypeId = storedInfoTypeId
        };

        // Call the API.
        StoredInfoType response = dlp.CreateStoredInfoType(request);

        // Inspect the response.
        Console.WriteLine($"Created the stored infotype at path: {response.Name}");

        return response;
    }
}

Go

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

import (
	"context"
	"fmt"
	"io"

	dlp "cloud.google.com/go/dlp/apiv2"
	"cloud.google.com/go/dlp/apiv2/dlppb"
)

// createStoredInfoType creates a custom stored info type based on your input data.
func createStoredInfoType(w io.Writer, projectID, outputPath string) error {
	// projectId := "my-project-id"
	// outputPath := "gs://" + "your-bucket-name" + "path/to/directory"

	ctx := context.Background()

	// Initialize a client once and reuse it to send multiple requests. Clients
	// are safe to use across goroutines. When the client is no longer needed,
	// call the Close method to cleanup its resources.
	client, err := dlp.NewClient(ctx)
	if err != nil {
		return err
	}

	// Closing the client safely cleans up background resources.
	defer client.Close()

	// Specify the name you want to give the dictionary.
	displayName := "Github Usernames"

	// Specify a description of the dictionary.
	description := "Dictionary of GitHub usernames used in commits"

	// Specify the path to the location in a Cloud Storage
	// bucket to store the created dictionary.
	cloudStoragePath := &dlppb.CloudStoragePath{
		Path: outputPath,
	}

	// Specify your term list is stored in BigQuery.
	bigQueryField := &dlppb.BigQueryField{
		Table: &dlppb.BigQueryTable{
			ProjectId: "bigquery-public-data",
			DatasetId: "samples",
			TableId:   "github_nested",
		},
		Field: &dlppb.FieldId{
			Name: "actor",
		},
	}

	// Specify the configuration of the large custom dictionary.
	largeCustomDictionaryConfig := &dlppb.LargeCustomDictionaryConfig{
		OutputPath: cloudStoragePath,
		Source: &dlppb.LargeCustomDictionaryConfig_BigQueryField{
			BigQueryField: bigQueryField,
		},
	}

	// Specify the configuration for stored infoType.
	storedInfoTypeConfig := &dlppb.StoredInfoTypeConfig{
		DisplayName: displayName,
		Description: description,
		Type: &dlppb.StoredInfoTypeConfig_LargeCustomDictionary{
			LargeCustomDictionary: largeCustomDictionaryConfig,
		},
	}

	// Combine configurations into a request for the service.
	req := &dlppb.CreateStoredInfoTypeRequest{
		Parent:           fmt.Sprintf("projects/%s/locations/global", projectID),
		Config:           storedInfoTypeConfig,
		StoredInfoTypeId: "github-usernames",
	}

	// Send the request and receive response from the service.
	resp, err := client.CreateStoredInfoType(ctx, req)
	if err != nil {
		return err
	}

	// Print the result.
	fmt.Fprintf(w, "output: %v", resp.Name)
	return nil

}

Java

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。


import com.google.cloud.dlp.v2.DlpServiceClient;
import com.google.privacy.dlp.v2.BigQueryField;
import com.google.privacy.dlp.v2.BigQueryTable;
import com.google.privacy.dlp.v2.CloudStoragePath;
import com.google.privacy.dlp.v2.CreateStoredInfoTypeRequest;
import com.google.privacy.dlp.v2.FieldId;
import com.google.privacy.dlp.v2.LargeCustomDictionaryConfig;
import com.google.privacy.dlp.v2.LocationName;
import com.google.privacy.dlp.v2.StoredInfoType;
import com.google.privacy.dlp.v2.StoredInfoTypeConfig;
import java.io.IOException;

public class CreateStoredInfoType {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.

    //The Google Cloud project id to use as a parent resource.
    String projectId = "your-project-id";
    // The path to the location in a GCS bucket to store the created dictionary.
    String outputPath = "gs://" + "your-bucket-name" + "path/to/directory";
    createStoredInfoType(projectId, outputPath);
  }

  // Creates a custom stored info type that contains GitHub usernames used in commits.
  public static void createStoredInfoType(String projectId, String outputPath)
      throws IOException {
    try (DlpServiceClient dlp = DlpServiceClient.create()) {

      // Optionally set a display name and a description.
      String displayName = "GitHub usernames";
      String description = "Dictionary of GitHub usernames used in commits";

      // The output path where the custom dictionary containing the GitHub usernames will be stored.
      CloudStoragePath cloudStoragePath =
          CloudStoragePath.newBuilder()
              .setPath(outputPath)
              .build();

      // The reference to the table containing the GitHub usernames.
      BigQueryTable table = BigQueryTable.newBuilder()
              .setProjectId("bigquery-public-data")
              .setDatasetId("samples")
              .setTableId("github_nested")
              .build();

      // The reference to the BigQuery field that contains the GitHub usernames.
      BigQueryField bigQueryField = BigQueryField.newBuilder()
              .setTable(table)
              .setField(FieldId.newBuilder().setName("actor").build())
              .build();

      LargeCustomDictionaryConfig largeCustomDictionaryConfig =
          LargeCustomDictionaryConfig.newBuilder()
              .setOutputPath(cloudStoragePath)
              .setBigQueryField(bigQueryField)
              .build();

      StoredInfoTypeConfig storedInfoTypeConfig = StoredInfoTypeConfig.newBuilder()
              .setDisplayName(displayName)
              .setDescription(description)
              .setLargeCustomDictionary(largeCustomDictionaryConfig)
              .build();

      // Combine configurations into a request for the service.
      CreateStoredInfoTypeRequest createStoredInfoType = CreateStoredInfoTypeRequest.newBuilder()
              .setParent(LocationName.of(projectId, "global").toString())
              .setConfig(storedInfoTypeConfig)
              .setStoredInfoTypeId("github-usernames")
              .build();

      // Send the request and receive response from the service.
      StoredInfoType response = dlp.createStoredInfoType(createStoredInfoType);

      // Print the results.
      System.out.println("Created Stored InfoType: " + response.getName());
    }
  }
}

Node.js

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

// Import the required libraries
const dlp = require('@google-cloud/dlp');

// Create a DLP client
const dlpClient = new dlp.DlpServiceClient();

// The project ID to run the API call under.
// const projectId = "your-project-id";

// The identifier for the stored infoType
// const infoTypeId = 'github-usernames';

// The path to the location in a Cloud Storage bucket to store the created dictionary
// const outputPath = 'cloud-bucket-path';

// The project ID the table is stored under
// This may or (for public datasets) may not equal the calling project ID
// const dataProjectId = 'my-project';

// The ID of the dataset to inspect, e.g. 'my_dataset'
// const datasetId = 'my_dataset';

// The ID of the table to inspect, e.g. 'my_table'
// const tableId = 'my_table';

// Field ID to be used for constructing dictionary
// const fieldName = 'field_name';

async function createStoredInfoType() {
  // The name you want to give the dictionary.
  const displayName = 'GitHub usernames';
  // A description of the dictionary.
  const description = 'Dictionary of GitHub usernames used in commits';

  // Specify configuration for the large custom dictionary
  const largeCustomDictionaryConfig = {
    outputPath: {
      path: outputPath,
    },
    bigQueryField: {
      table: {
        datasetId: datasetId,
        projectId: dataProjectId,
        tableId: tableId,
      },
      field: {
        name: fieldName,
      },
    },
  };

  // Stored infoType configuration that uses large custom dictionary.
  const storedInfoTypeConfig = {
    displayName: displayName,
    description: description,
    largeCustomDictionary: largeCustomDictionaryConfig,
  };

  // Construct the job creation request to be sent by the client.
  const request = {
    parent: `projects/${projectId}/locations/global`,
    config: storedInfoTypeConfig,
    storedInfoTypeId: infoTypeId,
  };

  // Send the job creation request and process the response.
  const [response] = await dlpClient.createStoredInfoType(request);

  // Print results
  console.log(`InfoType stored successfully: ${response.name}`);
}
await createStoredInfoType();

PHP

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。


use Google\Cloud\Dlp\V2\BigQueryField;
use Google\Cloud\Dlp\V2\BigQueryTable;
use Google\Cloud\Dlp\V2\Client\DlpServiceClient;
use Google\Cloud\Dlp\V2\CloudStoragePath;
use Google\Cloud\Dlp\V2\CreateStoredInfoTypeRequest;
use Google\Cloud\Dlp\V2\FieldId;
use Google\Cloud\Dlp\V2\LargeCustomDictionaryConfig;
use Google\Cloud\Dlp\V2\StoredInfoTypeConfig;

/**
 * Create a stored infoType.
 *
 * @param string $callingProjectId  The Google Cloud Project ID to run the API call under.
 * @param string $outputgcsPath     The path to the location in a Cloud Storage bucket to store the created dictionary.
 * @param string $storedInfoTypeId  The name of the custom stored info type.
 * @param string $displayName       The human-readable name to give the stored infoType.
 * @param string $description       A description for the stored infoType to be created.
 */
function create_stored_infotype(
    string $callingProjectId,
    string $outputgcsPath,
    string $storedInfoTypeId,
    string $displayName,
    string $description
): void {
    // Instantiate a client.
    $dlp = new DlpServiceClient();

    // The reference to the table containing the GitHub usernames.
    // The reference to the BigQuery field that contains the GitHub usernames.
    // Note: we have used public data
    $bigQueryField = (new BigQueryField())
        ->setTable((new BigQueryTable())
            ->setDatasetId('samples')
            ->setProjectId('bigquery-public-data')
            ->setTableId('github_nested'))
        ->setField((new FieldId())
            ->setName('actor'));

    $largeCustomDictionaryConfig = (new LargeCustomDictionaryConfig())
        // The output path where the custom dictionary containing the GitHub usernames will be stored.
        ->setOutputPath((new CloudStoragePath())
            ->setPath($outputgcsPath))
        ->setBigQueryField($bigQueryField);

    // Configure the StoredInfoType we want the service to perform.
    $storedInfoTypeConfig = (new StoredInfoTypeConfig())
        ->setDisplayName($displayName)
        ->setDescription($description)
        ->setLargeCustomDictionary($largeCustomDictionaryConfig);

    // Send the stored infoType creation request and process the response.
    $parent = "projects/$callingProjectId/locations/global";
    $createStoredInfoTypeRequest = (new CreateStoredInfoTypeRequest())
        ->setParent($parent)
        ->setConfig($storedInfoTypeConfig)
        ->setStoredInfoTypeId($storedInfoTypeId);
    $response = $dlp->createStoredInfoType($createStoredInfoTypeRequest);

    // Print results.
    printf('Successfully created Stored InfoType : %s', $response->getName());
}

Python

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

import google.cloud.dlp


def create_stored_infotype(
    project: str,
    stored_info_type_id: str,
    output_bucket_name: str,
) -> None:
    """Uses the Data Loss Prevention API to create stored infoType.
    Args:
        project: The Google Cloud project id to use as a parent resource.
        stored_info_type_id: The identifier for large custom dictionary.
        output_bucket_name: The name of the bucket in Google Cloud Storage
            that would store the created dictionary.
    """

    # Instantiate a client.
    dlp = google.cloud.dlp_v2.DlpServiceClient()

    # Construct the stored infoType Configuration dictionary. This example creates
    # a stored infoType from a term list stored in a publicly available BigQuery
    # database (bigquery-public-data.samples.github_nested).
    # The database contains all GitHub usernames used in commits.
    stored_info_type_config = {
        "display_name": "GitHub usernames",
        "description": "Dictionary of GitHub usernames used in commits",
        "large_custom_dictionary": {
            "output_path": {"path": f"gs://{output_bucket_name}"},
            # We can either use bigquery field or gcs file as a term list input option.
            "big_query_field": {
                "table": {
                    "project_id": "bigquery-public-data",
                    "dataset_id": "samples",
                    "table_id": "github_nested",
                },
                "field": {"name": "actor"},
            },
        },
    }

    # Convert the project id into a full resource id.
    parent = f"projects/{project}/locations/global"

    # Call the API.
    response = dlp.create_stored_info_type(
        request={
            "parent": parent,
            "config": stored_info_type_config,
            "stored_info_type_id": stored_info_type_id,
        }
    )

    # Print the result
    print(f"Created Stored InfoType: {response.name}")

REST

  1. 在 Cloud Storage bucket 中為字典建立新資料夾。Sensitive Data Protection 會在您指定的位置建立包含字典檔案的資料夾。
  2. 使用 storedInfoTypes.create 方法建立字典。create 方法會採用下列參數：
    • StoredInfoTypeConfig 物件，其中包含已儲存 infoType 的設定。包括：
      • description：字典說明。
      • displayName：您要為字典指定的名稱。
      • LargeCustomDictionaryConfig：包含大型自訂字典的設定。包括：
        • BigQueryField：如果字詞清單儲存在 BigQuery 中，請指定這個值。包含清單儲存位置的資料表參照，以及包含各個字典詞組的欄位。
        • CloudStorageFileSet：如果字詞清單儲存在 Cloud Storage，請指定這個項目。包含 Cloud Storage 中來源位置的網址，格式如下："gs://[PATH_TO_GS]"。支援萬用字元。
        • outputPath： Cloud Storage bucket 中用於儲存所建立字典的位置路徑。
    • storedInfoTypeId：已儲存 infoType 的 ID。重建、刪除或在檢查或去識別化工作中使用儲存的 infoType 時，您可以使用這個 ID 參照儲存的 infoType。如果將這個欄位留空，系統會為您產生 ID。

以下是 JSON 範例，傳送至 storedInfoTypes.create 方法時，會建立新的儲存 infoType，具體來說，就是大型自訂字典偵測工具。這個範例會從公開 BigQuery 資料庫 (bigquery-public-data.samples.github_nested) 中儲存的字詞清單，建立儲存的 infoType。該資料庫包含修訂版本中使用的所有 GitHub 使用者名稱。產生的字典輸出路徑會設為名為 dlptesting 的 Cloud Storage bucket，已儲存的 infoType 則命名為 github-usernames。

JSON 輸入

POST https://dlp.googleapis.com/v2/projects/PROJECT_ID/storedInfoTypes

{
  "config":{
    "displayName":"GitHub usernames",
    "description":"Dictionary of GitHub usernames used in commits",
    "largeCustomDictionary":{
      "outputPath":{
        "path":"gs://[PATH_TO_GS]"
      },
      "bigQueryField":{
        "table":{
          "datasetId":"samples",
          "projectId":"bigquery-public-data",
          "tableId":"github_nested"
        }
      }
    }
  },
  "storedInfoTypeId":"github-usernames"
}

重新建立字典

如要更新字典，請先更新來源字詞清單，然後指示 Sensitive Data Protection 重建儲存的 infoType。

  1. 更新 Cloud Storage 或 BigQuery 中的現有來源字詞清單。

    視需要新增、移除或變更字詞或詞組。

  2. 使用 Google Cloud 控制台或storedInfoTypes.patch方法「重建」已儲存的 infoType，建立新版本。

    重建作業會建立新版字典，並取代舊版字典。

將儲存的 infoType 重建為新版本時，舊版本會刪除。Sensitive Data Protection 更新儲存的 infoType 時，狀態會顯示為「pending」(待處理)。在此期間，舊版已儲存的 infoType 仍會存在。在儲存的 infoType 處於待處理狀態時執行的任何掃描，都會使用舊版儲存的 infoType 執行。

如要重建已儲存的 infoType，請按照下列步驟操作：

控制台

  1. 在 Cloud Storage 或 BigQuery 中更新並儲存字詞清單。

  2. 在 Google Cloud 控制台，前往已儲存的 infoType 清單。

    前往已儲存的 infoType

  3. 按一下要更新的儲存 infoType ID。

  4. 在「InfoType details」(InfoType 詳細資料) 頁面，按一下「Rebuild data」(重建資料)。

Sensitive Data Protection 會根據您對來源字詞清單所做的變更，重新建立儲存的 infoType。儲存的 infoType 狀態為「就緒」後，即可使用。任何使用已儲存 infoType 的範本或工作觸發條件，都會自動使用重建的版本。

C#

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。


using System;
using Google.Cloud.Dlp.V2;
using Google.Protobuf.WellKnownTypes;

public class UpdateStoredInfoTypes
{
    public static StoredInfoType Update(
        string gcsFileUri,
        string storedInfoTypePath,
        string outputPath)
    {
        // Instantiate the client.
        var dlp = DlpServiceClient.Create();

        // Construct the stored infotype config. Here, we will change the source from bigquery table to GCS file.
        var storedConfig = new StoredInfoTypeConfig
        {
            LargeCustomDictionary = new LargeCustomDictionaryConfig
            {
                CloudStorageFileSet = new CloudStorageFileSet
                {
                    Url = gcsFileUri
                },
                OutputPath = new CloudStoragePath
                {
                    Path = outputPath
                }
            }
        };

        // Construct the request using the stored config by specifying the update mask object
        // which represent the path of field to be updated.
        var request = new UpdateStoredInfoTypeRequest
        {
            Config = storedConfig,
            Name = storedInfoTypePath,
            UpdateMask = new FieldMask
            {
                Paths =
                {
                    "large_custom_dictionary.cloud_storage_file_set.url"
                }
            }
        };

        // Call the API.
        StoredInfoType response = dlp.UpdateStoredInfoType(request);

        // Inspect the result.
        Console.WriteLine(response);
        return response;
    }
}

Go

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

import (
	"context"
	"fmt"
	"io"

	dlp "cloud.google.com/go/dlp/apiv2"
	"cloud.google.com/go/dlp/apiv2/dlppb"
	"google.golang.org/protobuf/types/known/fieldmaskpb"
)

// updateStoredInfoType uses the Data Loss Prevention API to update stored infoType
// detector by changing the source term list from one stored in Bigquery
// to one stored in Cloud Storage.
func updateStoredInfoType(w io.Writer, projectID, gcsUri, fileSetUrl, infoTypeId string) error {
	// projectId := "your-project-id"
	// gcsUri := "gs://" + "your-bucket-name" + "/path/to/your/file.txt"
	// fileSetUrl := "your-cloud-storage-file-set"
	// infoTypeId := "your-stored-info-type-id"

	ctx := context.Background()

	// Initialize a client once and reuse it to send multiple requests. Clients
	// are safe to use across goroutines. When the client is no longer needed,
	// call the Close method to cleanup its resources.
	client, err := dlp.NewClient(ctx)
	if err != nil {
		return err
	}

	// Closing the client safely cleans up background resources.
	defer client.Close()

	// Set path in Cloud Storage.
	cloudStoragePath := &dlppb.CloudStoragePath{
		Path: gcsUri,
	}
	cloudStorageFileSet := &dlppb.CloudStorageFileSet{
		Url: fileSetUrl,
	}

	// Configuration for a custom dictionary created from a data source of any size
	largeCustomDictionaryConfig := &dlppb.LargeCustomDictionaryConfig{
		OutputPath: cloudStoragePath,
		Source: &dlppb.LargeCustomDictionaryConfig_CloudStorageFileSet{
			CloudStorageFileSet: cloudStorageFileSet,
		},
	}

	// Set configuration for stored infoTypes.
	storedInfoTypeConfig := &dlppb.StoredInfoTypeConfig{
		Type: &dlppb.StoredInfoTypeConfig_LargeCustomDictionary{
			LargeCustomDictionary: largeCustomDictionaryConfig,
		},
	}

	// Set mask to control which fields get updated.
	fieldMask := &fieldmaskpb.FieldMask{
		Paths: []string{"large_custom_dictionary.cloud_storage_file_set.url"},
	}
	// Construct the job creation request to be sent by the client.
	req := &dlppb.UpdateStoredInfoTypeRequest{
		Name:       fmt.Sprint("projects/" + projectID + "/storedInfoTypes/" + infoTypeId),
		Config:     storedInfoTypeConfig,
		UpdateMask: fieldMask,
	}

	// Use the client to send the API request.
	resp, err := client.UpdateStoredInfoType(ctx, req)
	if err != nil {
		return err
	}

	// Print the result.
	fmt.Fprintf(w, "output: %v", resp.Name)
	return nil
}

Java

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。


import com.google.cloud.dlp.v2.DlpServiceClient;
import com.google.privacy.dlp.v2.CloudStorageFileSet;
import com.google.privacy.dlp.v2.CloudStoragePath;
import com.google.privacy.dlp.v2.LargeCustomDictionaryConfig;
import com.google.privacy.dlp.v2.StoredInfoType;
import com.google.privacy.dlp.v2.StoredInfoTypeConfig;
import com.google.privacy.dlp.v2.StoredInfoTypeName;
import com.google.privacy.dlp.v2.UpdateStoredInfoTypeRequest;
import com.google.protobuf.FieldMask;
import java.io.IOException;

public class UpdateStoredInfoType {
  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    // The Google Cloud project id to use as a parent resource.
    String projectId = "your-project-id";
    // The path to file in GCS bucket that holds a collection of words and phrases to be searched by
    // the new infoType detector.
    String filePath = "gs://" + "your-bucket-name" + "/path/to/your/file.txt";
    // The path to the location in a GCS bucket to store the created dictionary.
    String outputPath = "your-cloud-storage-file-set";
    // The name of the stored InfoType which is to be updated.
    String infoTypeId = "your-stored-info-type-id";
    updateStoredInfoType(projectId, filePath, outputPath, infoTypeId);
  }

  // Update the stored info type rebuilding the Custom dictionary.
  public static void updateStoredInfoType(
      String projectId, String filePath, String outputPath, String infoTypeId) throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DlpServiceClient dlp = DlpServiceClient.create()) {
      // Set path in Cloud Storage.
      CloudStoragePath cloudStoragePath = CloudStoragePath.newBuilder().setPath(outputPath).build();
      CloudStorageFileSet cloudStorageFileSet =
          CloudStorageFileSet.newBuilder().setUrl(filePath).build();

      // Configuration for a custom dictionary created from a data source of any size
      LargeCustomDictionaryConfig largeCustomDictionaryConfig =
          LargeCustomDictionaryConfig.newBuilder()
              .setOutputPath(cloudStoragePath)
              .setCloudStorageFileSet(cloudStorageFileSet)
              .build();

      // Set configuration for stored infoTypes.
      StoredInfoTypeConfig storedInfoTypeConfig =
          StoredInfoTypeConfig.newBuilder()
              .setLargeCustomDictionary(largeCustomDictionaryConfig)
              .build();

      // Set mask to control which fields get updated.
      // Refer https://protobuf.dev/reference/protobuf/google.protobuf/#field-mask for constructing the field mask paths.
      FieldMask fieldMask =
          FieldMask.newBuilder()
              .addPaths("large_custom_dictionary.cloud_storage_file_set.url")
              .build();

      // Construct the job creation request to be sent by the client.
      UpdateStoredInfoTypeRequest updateStoredInfoTypeRequest =
          UpdateStoredInfoTypeRequest.newBuilder()
              .setName(
                  StoredInfoTypeName.ofProjectStoredInfoTypeName(projectId, infoTypeId).toString())
              .setConfig(storedInfoTypeConfig)
              .setUpdateMask(fieldMask)
              .build();

      // Send the job creation request and process the response.
      StoredInfoType response = dlp.updateStoredInfoType(updateStoredInfoTypeRequest);

      // Print the results.
      System.out.println("Updated stored InfoType successfully: " + response.getName());
    }
  }
}

Node.js

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

// Import the required libraries
const dlp = require('@google-cloud/dlp');

// Create a DLP client
const dlpClient = new dlp.DlpServiceClient();

// The project ID to run the API call under.
// const projectId = "your-project-id";

// The identifier for the stored infoType
// const infoTypeId = 'github-usernames';

// The path to the location in a Cloud Storage bucket to store the created dictionary
// const outputPath = 'cloud-bucket-path';

// Path of file containing term list
// const cloudStorageFileSet = 'gs://[PATH_TO_GS]';

async function updateStoredInfoType() {
  // Specify configuration of the large custom dictionary including cloudStorageFileSet and outputPath
  const largeCustomDictionaryConfig = {
    outputPath: {
      path: outputPath,
    },
    cloudStorageFileSet: {
      url: fileSetUrl,
    },
  };

  // Construct the job creation request to be sent by the client.
  const updateStoredInfoTypeRequest = {
    name: `projects/${projectId}/storedInfoTypes/${infoTypeId}`,
    config: {
      largeCustomDictionary: largeCustomDictionaryConfig,
    },
    updateMask: {
      paths: ['large_custom_dictionary.cloud_storage_file_set.url'],
    },
  };

  // Send the job creation request and process the response.
  const [response] = await dlpClient.updateStoredInfoType(
    updateStoredInfoTypeRequest
  );

  // Print the results.
  console.log(`InfoType updated successfully: ${JSON.stringify(response)}`);
}
await updateStoredInfoType();

PHP

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

use Google\Cloud\Dlp\V2\Client\DlpServiceClient;
use Google\Cloud\Dlp\V2\CloudStorageFileSet;
use Google\Cloud\Dlp\V2\CloudStoragePath;
use Google\Cloud\Dlp\V2\LargeCustomDictionaryConfig;
use Google\Cloud\Dlp\V2\StoredInfoTypeConfig;
use Google\Cloud\Dlp\V2\UpdateStoredInfoTypeRequest;
use Google\Protobuf\FieldMask;

/**
 * Rebuild/Update the stored infoType.
 *
 * @param string $callingProjectId  The Google Cloud Project ID to run the API call under.
 * @param string $gcsPath           The path to file in GCS bucket that holds a collection of words and phrases to be searched by the new infoType detector.
 * @param string $outputgcsPath     The path to the location in a Cloud Storage bucket to store the created dictionary.
 * @param string $storedInfoTypeId  The name of the stored InfoType which is to be updated.
 *
 */
function update_stored_infotype(
    string $callingProjectId,
    string $gcsPath,
    string $outputgcsPath,
    string $storedInfoTypeId
): void {
    // Instantiate a client.
    $dlp = new DlpServiceClient();

    // Set path in Cloud Storage.
    $cloudStorageFileSet = (new CloudStorageFileSet())
        ->setUrl($gcsPath);

    // Configuration for a custom dictionary created from a data source of any size
    $largeCustomDictionaryConfig = (new LargeCustomDictionaryConfig())
        ->setOutputPath((new CloudStoragePath())
            ->setPath($outputgcsPath))
        ->setCloudStorageFileSet($cloudStorageFileSet);

    // Set configuration for stored infoTypes.
    $storedInfoTypeConfig = (new StoredInfoTypeConfig())
        ->setLargeCustomDictionary($largeCustomDictionaryConfig);

    // Send the stored infoType creation request and process the response.

    $name = "projects/$callingProjectId/locations/global/storedInfoTypes/" . $storedInfoTypeId;
    // Set mask to control which fields get updated.
    // Refer https://protobuf.dev/reference/protobuf/google.protobuf/#field-mask for constructing the field mask paths.
    $fieldMask = (new FieldMask())
        ->setPaths([
            'large_custom_dictionary.cloud_storage_file_set.url'
        ]);

    // Run request
    $updateStoredInfoTypeRequest = (new UpdateStoredInfoTypeRequest())
        ->setName($name)
        ->setConfig($storedInfoTypeConfig)
        ->setUpdateMask($fieldMask);
    $response = $dlp->updateStoredInfoType($updateStoredInfoTypeRequest);

    // Print results
    printf('Successfully update Stored InforType : %s' . PHP_EOL, $response->getName());
}

Python

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

import google.cloud.dlp


def update_stored_infotype(
    project: str,
    stored_info_type_id: str,
    gcs_input_file_path: str,
    output_bucket_name: str,
) -> None:
    """Uses the Data Loss Prevention API to update stored infoType
    detector by changing the source term list from one stored in Bigquery
    to one stored in Cloud Storage.
    Args:
        project: The Google Cloud project id to use as a parent resource.
        stored_info_type_id: The identifier of stored infoType which is to
            be updated.
        gcs_input_file_path: The url in the format <bucket>/<path_to_file>
            for the location of the source term list.
        output_bucket_name: The name of the bucket in Google Cloud Storage
            where large dictionary is stored.
    """

    # Instantiate a client.
    dlp = google.cloud.dlp_v2.DlpServiceClient()

    # Construct the stored infoType configuration dictionary.
    stored_info_type_config = {
        "large_custom_dictionary": {
            "output_path": {"path": f"gs://{output_bucket_name}"},
            "cloud_storage_file_set": {"url": f"gs://{gcs_input_file_path}"},
        }
    }

    # Set mask to control which fields get updated. For more details, refer
    # https://protobuf.dev/reference/protobuf/google.protobuf/#field-mask
    # for constructing the field mask paths.
    field_mask = {"paths": ["large_custom_dictionary.cloud_storage_file_set.url"]}

    # Convert the stored infoType id into a full resource id.
    stored_info_type_name = (
        f"projects/{project}/locations/global/storedInfoTypes/{stored_info_type_id}"
    )

    # Call the API.
    response = dlp.update_stored_info_type(
        request={
            "name": stored_info_type_name,
            "config": stored_info_type_config,
            "update_mask": field_mask,
        }
    )

    # Print the result
    print(f"Updated stored infoType successfully: {response.name}")

REST

更新字詞清單

如果只要更新大型自訂字典中的字詞清單，storedInfoTypes.patch 要求只需包含 name 欄位。提供要重建的儲存 infoType 完整資源名稱。

下列模式代表 name 欄位的有效項目：

  • organizations/ORGANIZATION_ID/storedInfoTypes/STORED_INFOTYPE_ID
  • projects/PROJECT_ID/storedInfoTypes/STORED_INFOTYPE_ID

將 STORED_INFOTYPE_ID 替換為要重建的儲存 infoType ID。

如果您不知道已儲存 infoType 的 ID，請呼叫 storedInfoTypes.list 方法，查看目前所有已儲存 infoType 的清單。

範例

PATCH https://dlp.googleapis.com/v2/projects/PROJECT_ID/storedInfoTypes/STORED_INFOTYPE_ID

在這種情況下，不需要要求主體。

切換原文詞彙清單

您可以將已儲存 infoType 的來源字詞清單，從儲存在 BigQuery 的清單變更為儲存在 Cloud Storage 的清單。請使用 storedInfoTypes.patch 方法，但在 LargeCustomDictionaryConfig 中加入 CloudStorageFileSet 物件，而非先前使用的 BigQueryField 物件。接著，將 updateMask 參數設為您重建的儲存 infoType 參數，格式為 FieldMask。舉例來說，下列 JSON 在 updateMask 參數中指出 Cloud Storage 路徑的網址已更新 (large_custom_dictionary.cloud_storage_file_set.url)：

範例

PATCH https://dlp.googleapis.com/v2/projects/PROJECT_ID/storedInfoTypes/github-usernames

{
  "config":{
    "largeCustomDictionary":{
      "cloudStorageFileSet":{
        "url":"gs://[BUCKET_NAME]/[PATH_TO_FILE]"
      }
    }
  },
  "updateMask":"large_custom_dictionary.cloud_storage_file_set.url"
}

同樣地，您可以將字詞清單從儲存在 BigQuery 資料表中的清單，切換為儲存在 Cloud Storage bucket 中的清單。

使用大型自訂字典偵測工具掃描內容

使用大型自訂字典偵測工具掃描內容，與使用其他自訂 infoType 偵測工具掃描內容類似。

這項程序假設您已有儲存的 infoType。詳情請參閱本頁的「建立儲存的 infoType」一節。

控制台

執行下列操作時，可以套用大型自訂字典偵測工具：

在頁面的「Configure detection」(設定偵測) 部分，您可以在「InfoTypes」(infoType) 子節中指定大型自訂字典 infoType。

  1. 按一下「管理 infoType」。
  2. 在「InfoTypes」(infoType) 窗格中，按一下「Custom」(自訂) 分頁標籤。
  3. 按一下「Add custom infoType」(新增自訂 infoType)。

  4. 在「Add custom infoType」(新增自訂 infoType) 窗格中，執行下列操作：

    1. 在「Type」(類型) 部分，選取「Stored infoType」(已儲存的 infoType)。
    2. 在「InfoType」InfoType部分，輸入自訂 infoType 的名稱。可以使用英文字母、數字和底線。

    3. 在「可能性」部分，選取要指派給所有符合這項自訂 infoType 的發現項目的預設可能性等級。您可以使用啟動字詞規則，進一步微調個別發現項目的可能性等級。

      如未指定預設值，預設可能性層級會設為 VERY_LIKELY。詳情請參閱「相符可能性」。

    4. 在「Sensitivity」(機密程度) 部分，選取要指派給所有符合這項自訂 infoType 的發現項目的機密程度。如未指定值，這些發現項目的機密層級會設為 HIGH。

      資料剖析會使用敏感度分數。剖析資料時，Sensitive Data Protection 會使用 infoType 的機密程度分數計算機密程度。

    5. 在「Stored infoType name」(儲存的 infoType 名稱) 部分，選取要用來建立新自訂 infoType 的儲存 infoType。

    6. 按一下「Done」(完成)，關閉「Add custom infoType」(新增自訂 infoType) 窗格。

  5. 選用：在「Built-in」(內建) 分頁中，編輯選取的內建 infoType。

  6. 按一下「Done」(完成)，關閉「InfoTypes」(infoType) 窗格。

    自訂 infoType 會新增至 Sensitive Data Protection 掃描的 infoType 清單。不過，在您儲存工作、工作觸發條件、範本或掃描設定之前，這項選取項目並非最終結果。

  7. 設定建立或編輯完成後，請按一下「儲存」。

C#

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。


using System;
using Google.Api.Gax.ResourceNames;
using Google.Cloud.Dlp.V2;

public class InspectDataWithStoredInfotypes
{
    public static InspectContentResponse Inspect(
        string projectId,
        string storedInfotypePath,
        string text,
        InfoType infoType = null)
    {
        // Instantiate the dlp client.
        var dlp = DlpServiceClient.Create();

        // Construct the infotype if null.
        var infotype = infoType ?? new InfoType { Name = "GITHUB_LOGINS" };

        // Construct the inspect config using stored infotype.
        var inspectConfig = new InspectConfig
        {
            CustomInfoTypes =
            {
                new CustomInfoType
                {
                    InfoType = infotype,
                    StoredType = new StoredType { Name = storedInfotypePath }
                }
            },
            IncludeQuote = true
        };

        // Construct the request using inspect config.
        var request = new InspectContentRequest
        {
            ParentAsLocationName = new LocationName(projectId, "global"),
            InspectConfig = inspectConfig,
            Item = new ContentItem { Value = text }
        };

        // Call the API.
        InspectContentResponse response = dlp.InspectContent(request);

        // Inspect the results.
        var findings = response.Result.Findings;
        Console.WriteLine($"Findings: {findings.Count}");
        foreach (var f in findings)
        {
            Console.WriteLine("\tQuote: " + f.Quote);
            Console.WriteLine("\tInfo type: " + f.InfoType.Name);
            Console.WriteLine("\tLikelihood: " + f.Likelihood);
        }

        return response;
    }
}

Go

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

import (
	"context"
	"fmt"
	"io"

	dlp "cloud.google.com/go/dlp/apiv2"
	"cloud.google.com/go/dlp/apiv2/dlppb"
)

// inspectWithStoredInfotype inspects the given text using the specified stored infoType detector.
func inspectWithStoredInfotype(w io.Writer, projectID, infoTypeId, textToDeidentify string) error {
	// projectId := "your-project-id"
	// infoTypeId := "your-info-type-id"
	// textToDeidentify := "This commit was made by kewin2010"

	ctx := context.Background()

	// Initialize a client once and reuse it to send multiple requests. Clients
	// are safe to use across goroutines. When the client is no longer needed,
	// call the Close method to cleanup its resources.
	client, err := dlp.NewClient(ctx)
	if err != nil {
		return err
	}

	// Closing the client safely cleans up background resources.
	defer client.Close()

	// Specify the content to be inspected.
	contentItem := &dlppb.ContentItem{
		DataItem: &dlppb.ContentItem_Value{
			Value: textToDeidentify,
		},
	}

	// Specify the info type the inspection will look for.
	infoType := &dlppb.InfoType{
		Name: "GITHUB_LOGINS",
	}

	// Specify the stored info type the inspection will look for.
	storedType := &dlppb.StoredType{
		Name: infoTypeId,
	}

	customInfoType := &dlppb.CustomInfoType{
		InfoType: infoType,
		Type: &dlppb.CustomInfoType_StoredType{
			StoredType: storedType,
		},
	}

	// Specify how the content should be inspected.
	inspectConfig := &dlppb.InspectConfig{
		CustomInfoTypes: []*dlppb.CustomInfoType{
			customInfoType,
		},
		IncludeQuote: true,
	}

	// Construct the Inspect request to be sent by the client.
	req := &dlppb.InspectContentRequest{
		Parent:        fmt.Sprintf("projects/%s/locations/global", projectID),
		InspectConfig: inspectConfig,
		Item:          contentItem,
	}

	// Use the client to send the API request.
	resp, err := client.InspectContent(ctx, req)
	if err != nil {
		return err
	}

	// Process the results.
	fmt.Fprintf(w, "Findings: %d\n", len(resp.Result.Findings))
	for _, f := range resp.Result.Findings {
		fmt.Fprintf(w, "\tQuote: %s\n", f.Quote)
		fmt.Fprintf(w, "\tInfo type: %s\n", f.InfoType.Name)
		fmt.Fprintf(w, "\tLikelihood: %s\n", f.Likelihood)
	}
	return nil
}

Java

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。


import com.google.cloud.dlp.v2.DlpServiceClient;
import com.google.privacy.dlp.v2.ContentItem;
import com.google.privacy.dlp.v2.CustomInfoType;
import com.google.privacy.dlp.v2.Finding;
import com.google.privacy.dlp.v2.InfoType;
import com.google.privacy.dlp.v2.InspectConfig;
import com.google.privacy.dlp.v2.InspectContentRequest;
import com.google.privacy.dlp.v2.InspectContentResponse;
import com.google.privacy.dlp.v2.LocationName;
import com.google.privacy.dlp.v2.ProjectStoredInfoTypeName;
import com.google.privacy.dlp.v2.StoredType;
import java.io.IOException;

public class InspectWithStoredInfotype {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    // The Google Cloud project id to use as a parent resource.
    String projectId = "your-project-id";
    // The sample assumes that you have an existing stored infoType.
    // To create a stored InfoType refer:
    // https://cloud.google.com/dlp/docs/creating-stored-infotypes#create-storedinfotye 
    String storedInfoTypeId = "your-info-type-id";
    // The string to de-identify.
    String textToInspect =
        "My phone number is (223) 456-7890 and my email address is gary@example.com.";
    inspectWithStoredInfotype(projectId, storedInfoTypeId, textToInspect);
  }

  //  Inspects the given text using the specified stored infoType detector.
  public static void inspectWithStoredInfotype(
      String projectId, String storedInfoTypeId, String textToInspect) throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DlpServiceClient dlp = DlpServiceClient.create()) {

      // Specify the content to be inspected.
      ContentItem contentItem = ContentItem.newBuilder().setValue(textToInspect).build();

      InfoType infoType = InfoType.newBuilder().setName("STORED_TYPE").build();

      // Reference to the existing StoredInfoType to inspect the data.
      StoredType storedType = StoredType.newBuilder()
              .setName(ProjectStoredInfoTypeName.of(projectId, storedInfoTypeId).toString())
              .build();

      CustomInfoType customInfoType =
          CustomInfoType.newBuilder().setInfoType(infoType).setStoredType(storedType).build();

      // Construct the configuration for the Inspect request.
      InspectConfig inspectConfig =
          InspectConfig.newBuilder()
              .addCustomInfoTypes(customInfoType)
              .setIncludeQuote(true)
              .build();

      // Construct the Inspect request to be sent by the client.
      InspectContentRequest inspectContentRequest =
          InspectContentRequest.newBuilder()
              .setParent(LocationName.of(projectId, "global").toString())
              .setInspectConfig(inspectConfig)
              .setItem(contentItem)
              .build();

      // Use the client to send the API request.
      InspectContentResponse response = dlp.inspectContent(inspectContentRequest);

      // Parse the response and process results.
      System.out.println("Findings: " + "" + response.getResult().getFindingsCount());
      for (Finding f : response.getResult().getFindingsList()) {
        System.out.println("\tQuote: " + f.getQuote());
        System.out.println("\tInfoType: " + f.getInfoType().getName());
        System.out.println("\tLikelihood: " + f.getLikelihood() + "\n");
      }
    }
  }
}

Node.js

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

// Imports the Google Cloud Data Loss Prevention library
const DLP = require('@google-cloud/dlp');

// Instantiates a client
const dlp = new DLP.DlpServiceClient();

// The project ID to run the API call under.
// const projectId = 'your-project-id';

// The custom info-type id created and stored in the bucket.
// const infoTypeId = 'your-info-type-id';

// The string to inspect.
// const string = 'My phone number is (223) 456-7890 and my email address is gary@example.com.';

async function inspectWithStoredInfotype() {
  // Reference to the existing StoredInfoType to inspect the data.
  const customInfoType = {
    infoType: {
      name: 'GITHUB_LOGINS',
    },
    storedType: {
      name: infoTypeId,
    },
  };

  // Construct the configuration for the Inspect request.
  const inspectConfig = {
    customInfoTypes: [customInfoType],
    includeQuote: true,
  };

  // Construct the Inspect request to be sent by the client.
  const request = {
    parent: `projects/${projectId}/locations/global`,
    inspectConfig: inspectConfig,
    item: {
      value: string,
    },
  };
  // Run request
  const [response] = await dlp.inspectContent(request);

  // Print Findings
  const findings = response.result.findings;
  if (findings.length > 0) {
    console.log(`Findings: ${findings.length}\n`);
    findings.forEach(finding => {
      console.log(`InfoType: ${finding.infoType.name}`);
      console.log(`\tQuote: ${finding.quote}`);
      console.log(`\tLikelihood: ${finding.likelihood} \n`);
    });
  } else {
    console.log('No findings.');
  }
}
await inspectWithStoredInfotype();

PHP

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

use Google\Cloud\Dlp\V2\Client\DlpServiceClient;
use Google\Cloud\Dlp\V2\ContentItem;
use Google\Cloud\Dlp\V2\CustomInfoType;
use Google\Cloud\Dlp\V2\InfoType;
use Google\Cloud\Dlp\V2\InspectConfig;
use Google\Cloud\Dlp\V2\InspectContentRequest;
use Google\Cloud\Dlp\V2\Likelihood;
use Google\Cloud\Dlp\V2\StoredType;

/**
 * Inspect with stored infoType.
 * Scan content using a large custom dictionary detector.
 *
 * @param string $projectId             The Google Cloud Project ID to run the API call under.
 * @param string $storedInfoTypeName    The name of the stored infotype whose This value must be in the format
 * projects/projectName/(locations/locationId)/storedInfoTypes/storedInfoTypeName.
 * @param string $textToInspect         The string to inspect.
 */
function inspect_with_stored_infotype(
    string $projectId,
    string $storedInfoTypeName,
    string $textToInspect
): void {
    // Instantiate a client.
    $dlp = new DlpServiceClient();

    $parent = "projects/$projectId/locations/global";

    // Specify the content to be inspected.
    $item = (new ContentItem())
        ->setValue($textToInspect);

    // Reference to the existing StoredInfoType to inspect the data.
    $customInfoType = (new CustomInfoType())
        ->setInfoType((new InfoType())
            ->setName('STORED_TYPE'))
        ->setStoredType((new StoredType())
            ->setName($storedInfoTypeName));

    // Construct the configuration for the Inspect request.
    $inspectConfig = (new InspectConfig())
        ->setCustomInfoTypes([$customInfoType])
        ->setIncludeQuote(true);

    // Run request.
    $inspectContentRequest = (new InspectContentRequest())
        ->setParent($parent)
        ->setInspectConfig($inspectConfig)
        ->setItem($item);
    $response = $dlp->inspectContent($inspectContentRequest);

    // Print the results.
    $findings = $response->getResult()->getFindings();
    if (count($findings) == 0) {
        printf('No findings.' . PHP_EOL);
    } else {
        printf('Findings:' . PHP_EOL);
        foreach ($findings as $finding) {
            printf('  Quote: %s' . PHP_EOL, $finding->getQuote());
            printf('  Info type: %s' . PHP_EOL, $finding->getInfoType()->getName());
            printf('  Likelihood: %s' . PHP_EOL, Likelihood::name($finding->getLikelihood()));
        }
    }
}

Python

如要瞭解如何安裝及使用 Sensitive Data Protection 的用戶端程式庫，請參閱「Sensitive Data Protection 用戶端程式庫」。

如要向 Sensitive Data Protection 進行驗證，請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。

import google.cloud.dlp


def inspect_with_stored_infotype(
    project: str,
    stored_info_type_id: str,
    content_string: str,
) -> None:
    """Uses the Data Loss Prevention API to inspect/scan content using stored
    infoType.
    Args:
        project: The Google Cloud project id to use as a parent resource.
        content_string: The string to inspect.
        stored_info_type_id: The identifier of stored infoType used to inspect.
    """

    # Instantiate a client.
    dlp = google.cloud.dlp_v2.DlpServiceClient()

    # Convert stored infoType id into full resource id
    stored_type_name = f"projects/{project}/storedInfoTypes/{stored_info_type_id}"

    # Construct a custom info type dictionary using stored infoType.
    custom_info_types = [
        {
            "info_type": {"name": "STORED_TYPE"},
            "stored_type": {
                "name": stored_type_name,
            },
        }
    ]

    # Construct the inspection configuration dictionary.
    inspect_config = {
        "custom_info_types": custom_info_types,
        "include_quote": True,
    }

    # Construct the `item` to be inspected using stored infoType.
    item = {"value": content_string}

    # Convert the project id into a full resource id.
    parent = f"projects/{project}/locations/global"

    # Call the API.
    response = dlp.inspect_content(
        request={
            "parent": parent,
            "inspect_config": inspect_config,
            "item": item,
        }
    )

    # Print out the results.
    if response.result.findings:
        for finding in response.result.findings:
            print(f"Quote: {finding.quote}")
            print(f"Info type: {finding.info_type.name}")
            print(f"Likelihood: {finding.likelihood}")
    else:
        print("No findings.")

REST

傳送至 content.inspect 方法時，下列範例會使用指定的儲存 infoType 偵測工具掃描指定文字。infoType 參數為必填，因為所有自訂 infoType 都必須有名稱，且不得與內建 infoType 或其他自訂 infoType 衝突。storedType 參數包含已儲存 infoType 的完整資源路徑。

JSON 輸入

POST https://dlp.googleapis.com/v2/projects/PROJECT_ID/content:inspect

{
  "inspectConfig":{
    "customInfoTypes":[
      {
        "infoType":{
          "name":"GITHUB_LOGINS"
        },
        "storedType":{
          "name":"projects/PROJECT_ID/storedInfoTypes/github-logins"
        }
      }
    ]
  },
  "item":{
    "value":"The commit was made by githubuser."
  }
}

排解錯誤

如果嘗試從 Cloud Storage 中儲存的字詞清單建立儲存的 infoType 時發生錯誤，可能原因如下：

  • 您已達到儲存 infoType 的上限。視問題而定，有幾種解決方法：
    • 如果 Cloud Storage 中單一輸入檔案的大小達到上限 (200 MB)，請嘗試將檔案分割成多個檔案。您可以使用多個檔案來組合單一自訂字典，只要所有檔案的合併大小不超過 1 GB 即可。
    • BigQuery 的限制與 Cloud Storage 不同。建議將字詞移至 BigQuery 資料表。BigQuery 中自訂字典資料欄的大小上限為 1 GB，資料列數量上限為 5,000,000。
    • 如果字詞清單檔案超過來源字詞清單的所有適用限制，您必須將字詞清單檔案分割成多個檔案，並為每個檔案建立字典。接著，為每個字典建立個別的掃描工作。
  • 一或多個字詞不含任何字母或數字。 Sensitive Data Protection 無法掃描僅由空格或符號組成的字詞。字詞必須至少有一個字母或數字。檢查您的字詞清單，看看是否含有這類的字詞，然後予以修正或刪除。
  • 字詞清單中的詞組含有過多「組成部分」。這裡的「成分」是指一連串不間斷的純字母、純數字，或純非字母和非數字字元 (例如空格或符號)。檢查您的字詞清單，看看是否含有這類的字詞，然後予以修正或刪除。
  • Sensitive Data Protection 服務代理無法存取字典來源資料，也無法存取用於儲存字典檔案的 Cloud Storage bucket。如要修正這個問題，請將 Storage 管理員 (roles/storage.admin) 角色，或 BigQuery 資料擁有者 (roles/bigquery.dataOwner) 和 BigQuery 工作使用者 (roles/bigquery.jobUser) 角色，授予 Sensitive Data Protection 服務代理。

API 總覽

如要建立大型自訂字典偵測工具，必須建立儲存的 infoType。

在 Sensitive Data Protection 中，儲存的 infoType 是以 StoredInfoType 物件表示。包含下列相關物件：

  • StoredInfoTypeVersion 包含建立日期和時間，以及建立目前版本時發生的最後五則錯誤訊息。

    • StoredInfoTypeConfig 包含儲存的 infoType 設定，包括名稱和說明。如果是大型自訂字典，type 必須是 LargeCustomDictionaryConfig。

      • LargeCustomDictionaryConfig 會指定下列兩項：
        • 詞組清單儲存在 Cloud Storage 或 BigQuery 的位置。
        • Cloud Storage 儲存產生的字典檔案的位置。

    • StoredInfoTypeState 包含已儲存 infoType 的最新版本和任何待處理版本的狀態。狀態資訊包括儲存的 infoType 是否正在重建、可供使用或無效。

字典比對細節

以下說明 Sensitive Data Protection 如何比對字典字詞和詞組。以下幾點適用於一般和大型自訂字典：

  • 字典的字詞不區分大小寫。如果字典包含 Abby，則會比對 abby、ABBY、Abby 等。
  • 在掃描比對時，字典或要掃描的內容中，除了 Unicode 基本多語言平面中的字母、數字和其他字母字元外，所有字元都會視為空白字元。如果字典掃描 Abby Abernathy，則會比對 abby abernathy、Abby, Abernathy、Abby (ABERNATHY) 等。
  • 任何相符項目周圍的字元必須是不同類型 (字母或數字)，不可以是字詞中的相鄰字元。如果字典掃描 Abi，則會比對 Abi904 的前三個字元，但不會比對 Abigail。
  • 如果字典字詞包含 Unicode 標準輔助多語言平面中的字元，可能會產生非預期的結果。這類字元的例子包括表情符號、科學符號和歷史文字。

字母、數字和其他字母字元的定義如下：

  • 字母：萬國碼 (Unicode) 規格中一般類別為 Lu、Ll、Lt、Lm 或 Lo 的字元
  • 數字：Unicode 規格中一般類別為 Nd 的字元
  • 其他英文字母字元：Unicode 規格中一般類別為 Nl 的字元，或 Unicode 標準定義中輔助屬性為 Other_Alphabetic 的字元

如要建立、編輯或刪除儲存的 infoType，請使用以下方法：

  • storedInfoTypes.create：根據您指定的 StoredInfoTypeConfig 建立新的儲存 infoType。
  • storedInfoTypes.patch：使用您指定的新 StoredInfoTypeConfig 重建已儲存的 infoType。如未指定，這個方法會使用現有的 StoredInfoTypeConfig 建立新版儲存的 infoType。
  • storedInfoTypes.get：擷取 StoredInfoTypeConfig 和指定已儲存 infoType 的任何待處理版本。
  • storedInfoTypes.list：列出目前儲存的所有 infoType。
  • storedInfoTypes.delete：刪除指定的已儲存 infoType。