使用模擬 Cloud Storage 用戶端編寫測試

Rust 適用的 Cloud 用戶端程式庫提供一種方法，可虛設常式實際用戶端實作，以便您插入模擬進行測試。您可以使用模擬物件，編寫可控且可靠的單元測試，不會發出網路呼叫或產生費用。

本文將說明下列主題：

  • 如何使用 Cloud Storage Storage 用戶端編寫可測試的介面。
  • 如何模擬讀取作業。
  • 如何模擬寫入作業。
  • Storage 用戶端的設計與其他 Google Cloud用戶端不同的原因。

依附元件

將 Cloud Storage 用戶端程式庫新增至 Cargo.toml 檔案：

cargo add google-cloud-storage

如要編譯並執行本文件中的單元測試，請新增下列開發依附元件：

cargo add --dev anyhow bytes mockall
cargo add --dev tokio --features macros,rt-multi-thread

編寫可測試的介面

如果您不對儲存空間程式碼進行單元測試，可以直接根據 Storage 編寫介面，其預設虛設常式類型參數是用戶端的正式版實作：

pub async fn my_function(_client: Storage) {}

如要使用模擬用戶端測試程式碼，請根據受 gcs::stub::Storage + 'static 約束的一般類型參數 T 編寫介面：

pub async fn my_testable_function<T>(_client: Storage<T>)
where
    T: gcs::stub::Storage + 'static,
{
}

模擬讀取作業

如要模擬 read_object 要求，請完成下列步驟：

  1. 編寫採用一般 &Storage<T> 用戶端的應用程式函式。舉例來說，下列函式會下載物件，並計算其中包含的換行字元數量： 

    // Downloads an object from GCS and counts the total lines.
pub async fn count_newlines<T>(
    client: &Storage<T>,
    bucket_id: &str,
    object_id: &str,
) -> gcs::Result<usize>
where
    T: gcs::stub::Storage + 'static,
{
    let mut count = 0;
    let mut reader = client
        .read_object(format!("projects/_/buckets/{bucket_id}"), object_id)
        .set_generation(42)
        .send()
        .await?;
    while let Some(buffer) = reader.next().await.transpose()? {
        count += buffer.into_iter().filter(|c| *c == b'\n').count();
    }
    Ok(count)
}

  2. 建立輔助函式，透過將酬載傳遞至 ReadObjectResponse::from_source，建構假的 ReadObjectResponse 位元組串流。這個方法接受與 Storage::write_object 相同的酬載類型： 

    fn fake_response(size: usize) -> ReadObjectResponse {
    let mut contents = String::new();
    for i in 0..size {
        contents.push_str(&format!("{i}\n"))
    }
    ReadObjectResponse::from_source(ObjectHighlights::default(), bytes::Bytes::from(contents))
}

  3. 定義實作 gcs::stub::Storage 的模擬結構體。本文使用 mockall crate，但您可以在測試中使用其他模擬架構：

    mockall::mock! {
    #[derive(Debug)]
    Storage {}
    impl gcs::stub::Storage for Storage {
        async fn read_object(&self, _req: ReadObjectRequest, _options: RequestOptions) -> Result<ReadObjectResponse>;
        async fn write_object_buffered<P: StreamingSource + Send + Sync + 'static>(
            &self,
            _payload: P,
            _req: WriteObjectRequest,
            _options: RequestOptions,
        ) -> Result<Object>;
        async fn write_object_unbuffered<P: StreamingSource + Seek + Send + Sync + 'static>(
            &self,
            _payload: P,
            _req: WriteObjectRequest,
            _options: RequestOptions,
        ) -> Result<Object>;
    }
}

  4. 編寫單元測試，設定模擬物件的預期行為、使用 Storage::from_stub 建構用戶端，並呼叫 count_newlines： 

    #[tokio::test]
async fn test_count_lines() -> anyhow::Result<()> {
    let mut mock = MockStorage::new();
    mock.expect_read_object().return_once({
        move |r, _| {
            // Verify contents of the request
            assert_eq!(r.generation, 42);
            assert_eq!(r.bucket, "projects/_/buckets/my-bucket");
            assert_eq!(r.object, "my-object");

            // Return a `ReadObjectResponse`
            Ok(fake_response(100))
        }
    });
    let client = gcs::client::Storage::from_stub(mock);

    let count = count_newlines(&client, "my-bucket", "my-object").await?;
    assert_eq!(count, 100);

    Ok(())
}

模擬寫入作業

如要模擬 write_object 要求，請完成下列步驟：

  1. 編寫應用程式函式，使用一般 &Storage<T> 用戶端上傳物件：

    // Uploads an object to GCS.
pub async fn upload<T>(client: &Storage<T>, bucket_id: &str, object_id: &str) -> gcs::Result<Object>
where
    T: gcs::stub::Storage + 'static,
{
    client
        .write_object(
            format!("projects/_/buckets/{bucket_id}"),
            object_id,
            "payload",
        )
        .set_if_generation_match(42)
        .send_unbuffered()
        .await
}

  2. 使用上一節定義的 MockStorage 結構，編寫單元測試，設定模擬物件的預期行為、使用 Storage::from_stub 建構用戶端，並呼叫 upload：

    #[tokio::test]
async fn test_upload() -> anyhow::Result<()> {
    let mut mock = MockStorage::new();
    mock.expect_write_object_unbuffered()
        .return_once(
            |_payload: Payload<BytesSource>, r, _| {
                // Verify contents of the request
                assert_eq!(r.spec.if_generation_match, Some(42));
                let o = r.spec.resource.unwrap_or_default();
                assert_eq!(o.bucket, "projects/_/buckets/my-bucket");
                assert_eq!(o.name, "my-object");

                // Return the object
                Ok(Object::default()
                    .set_bucket("projects/_/buckets/my-bucket")
                    .set_name("my-object")
                    .set_generation(42))
            },
        );
    let client = gcs::client::Storage::from_stub(mock);

    let object = upload(&client, "my-bucket", "my-object").await?;
    assert_eq!(object.generation, 42);

    Ok(())
}

預期詳細資料

由於 upload 函式會呼叫 send_unbuffered，因此請對應的 expect_write_object_unbuffered 方法設定預期：

mock.expect_write_object_unbuffered()

mockall::mock! 巨集會將每個一般例項化視為不同的方法。在閉包簽章中指定具體酬載類型 Payload<BytesSource>，讓編譯器能比對方法預期：

|_payload: Payload<BytesSource>, r, _| {

設計原理

以下各節說明 Storage 用戶端為何使用一般類型參數做為虛設常式，而非動態調度。

其他用戶端

大多數用戶端 (例如 StorageControl) 會在內部保留與 dyn 相容的虛設常式特徵實作，並採用參照計數 (Arc)。它們使用動態調度，將用戶端的要求轉送至底層虛設常式，該虛設常式可以是實際工作環境實作或模擬。

由於這些用戶端使用動態調度，編譯器不需要在編譯時間解析具體虛設常式類型，且用戶端結構體不需要虛設常式的泛型類型參數。

Storage 用戶端

在 Rust 中，與 dyn 相容的特徵不能有泛型方法，也不能傳回未封裝的 impl Future 值：

  • write_object_buffered 和 write_object_unbuffered 方法是酬載來源類型的泛型。
  • gcs::stub::Storage 特徵方法會直接傳回 impl Future，而不是將傳回的 Future 裝箱。

如果對 Storage 用戶端使用動態調度，則每次呼叫時都需要對酬載和 Future 進行裝箱，進而增加資料路徑上的堆積分配量和動態調度負擔。為避免這種負擔，Storage 用戶端會改用類型參數，具體實作 gcs::stub::Storage 特徵。

完整的應用程式程式碼和測試套件

下列程式碼將應用程式函式和單元測試結合成已完成樣本：

use gcs::client::Storage;
use gcs::model::Object;
use google_cloud_storage as gcs;

// Downloads an object from GCS and counts the total lines.
pub async fn count_newlines<T>(
    client: &Storage<T>,
    bucket_id: &str,
    object_id: &str,
) -> gcs::Result<usize>
where
    T: gcs::stub::Storage + 'static,
{
    let mut count = 0;
    let mut reader = client
        .read_object(format!("projects/_/buckets/{bucket_id}"), object_id)
        .set_generation(42)
        .send()
        .await?;
    while let Some(buffer) = reader.next().await.transpose()? {
        count += buffer.into_iter().filter(|c| *c == b'\n').count();
    }
    Ok(count)
}

// Uploads an object to GCS.
pub async fn upload<T>(client: &Storage<T>, bucket_id: &str, object_id: &str) -> gcs::Result<Object>
where
    T: gcs::stub::Storage + 'static,
{
    client
        .write_object(
            format!("projects/_/buckets/{bucket_id}"),
            object_id,
            "payload",
        )
        .set_if_generation_match(42)
        .send_unbuffered()
        .await
}

#[cfg(test)]
mod tests {
    use super::{count_newlines, upload};
    use gcs::Result;
    use gcs::model::{Object, ReadObjectRequest};
    use gcs::model_ext::{ObjectHighlights, WriteObjectRequest};
    use gcs::read_object::ReadObjectResponse;
    use gcs::request_options::RequestOptions;
    use gcs::streaming_source::{BytesSource, Payload, Seek, StreamingSource};
    use google_cloud_storage as gcs;

    mockall::mock! {
        #[derive(Debug)]
        Storage {}
        impl gcs::stub::Storage for Storage {
            async fn read_object(&self, _req: ReadObjectRequest, _options: RequestOptions) -> Result<ReadObjectResponse>;
            async fn write_object_buffered<P: StreamingSource + Send + Sync + 'static>(
                &self,
                _payload: P,
                _req: WriteObjectRequest,
                _options: RequestOptions,
            ) -> Result<Object>;
            async fn write_object_unbuffered<P: StreamingSource + Seek + Send + Sync + 'static>(
                &self,
                _payload: P,
                _req: WriteObjectRequest,
                _options: RequestOptions,
            ) -> Result<Object>;
        }
    }

    fn fake_response(size: usize) -> ReadObjectResponse {
        let mut contents = String::new();
        for i in 0..size {
            contents.push_str(&format!("{i}\n"))
        }
        ReadObjectResponse::from_source(ObjectHighlights::default(), bytes::Bytes::from(contents))
    }

    #[tokio::test]
    async fn test_count_lines() -> anyhow::Result<()> {
        let mut mock = MockStorage::new();
        mock.expect_read_object().return_once({
            move |r, _| {
                // Verify contents of the request
                assert_eq!(r.generation, 42);
                assert_eq!(r.bucket, "projects/_/buckets/my-bucket");
                assert_eq!(r.object, "my-object");

                // Return a `ReadObjectResponse`
                Ok(fake_response(100))
            }
        });
        let client = gcs::client::Storage::from_stub(mock);

        let count = count_newlines(&client, "my-bucket", "my-object").await?;
        assert_eq!(count, 100);

        Ok(())
    }

    #[tokio::test]
    async fn test_upload() -> anyhow::Result<()> {
        let mut mock = MockStorage::new();
        mock.expect_write_object_unbuffered()
            .return_once(
                |_payload: Payload<BytesSource>, r, _| {
                    // Verify contents of the request
                    assert_eq!(r.spec.if_generation_match, Some(42));
                    let o = r.spec.resource.unwrap_or_default();
                    assert_eq!(o.bucket, "projects/_/buckets/my-bucket");
                    assert_eq!(o.name, "my-object");

                    // Return the object
                    Ok(Object::default()
                        .set_bucket("projects/_/buckets/my-bucket")
                        .set_name("my-object")
                        .set_generation(42))
                },
            );
        let client = gcs::client::Storage::from_stub(mock);

        let object = upload(&client, "my-bucket", "my-object").await?;
        assert_eq!(object.generation, 42);

        Ok(())
    }
}

執行 cargo test，以執行 test_count_lines 和 test_upload：

cargo test