+ close()
Close httplib2 connections.
+
+ create(parent, body=None, x__xgafv=None)
Creates an Evaluation Item.
+ +Deletes an Evaluation Item.
+ +Gets an Evaluation Item.
+
+ list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
Lists Evaluation Items.
+ +Retrieves the next page of results.
+close()
+ Close httplib2 connections.+
create(parent, body=None, x__xgafv=None)
+ Creates an Evaluation Item.
+
+Args:
+ parent: string, Required. The resource name of the Location to create the Evaluation Item in. Format: `projects/{project}/locations/{location}` (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # EvaluationItem is a single evaluation request or result. The content of an EvaluationItem is immutable - it cannot be updated once created. EvaluationItems can be deleted when no longer needed.
+ "createTime": "A String", # Output only. Timestamp when this item was created.
+ "displayName": "A String", # Required. The display name of the EvaluationItem.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Output only. Error for the evaluation item.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "evaluationItemType": "A String", # Required. The type of the EvaluationItem.
+ "evaluationRequest": { # Single evaluation request. # The request to evaluate.
+ "candidateResponses": [ # Optional. Responses from model under test and other baseline models for comparison.
+ { # Responses from model or agent.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ ],
+ "goldenResponse": { # Responses from model or agent. # Optional. The Ideal response or ground truth.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ "prompt": { # Prompt to be evaluated. # Required. The request/prompt to evaluate.
+ "promptTemplateData": { # Message to hold a prompt template and the values to populate the template. # Prompt template data.
+ "values": { # The values for fields in the prompt template.
+ "a_key": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
+ "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
+ { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
+ "codeExecutionResult": { # Result of executing the [ExecutableCode]. Only generated when using the [CodeExecution] tool, and always follows a `part` containing the [ExecutableCode]. # Optional. Result of executing the [ExecutableCode].
+ "outcome": "A String", # Required. Outcome of the code execution.
+ "output": "A String", # Optional. Contains stdout when code execution is successful, stderr or other description otherwise.
+ },
+ "executableCode": { # Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated. # Optional. Code generated by the model that is meant to be executed.
+ "code": "A String", # Required. The code to be executed.
+ "language": "A String", # Required. Programming language of the `code`.
+ },
+ "fileData": { # URI based data. # Optional. URI based data.
+ "displayName": "A String", # Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "fileUri": "A String", # Required. URI.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
+ "args": { # Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
+ "a_key": "", # Properties of the object.
+ },
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
+ },
+ "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
+ "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "inlineData": { # Content blob. # Optional. Inlined bytes data.
+ "data": "A String", # Required. Raw bytes.
+ "displayName": "A String", # Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "text": "A String", # Optional. Text part (can be code).
+ "thought": True or False, # Optional. Indicates if the part is thought from the model.
+ "thoughtSignature": "A String", # Optional. An opaque signature for the thought so it can be reused in subsequent requests.
+ "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
+ "endOffset": "A String", # Optional. The end offset of the video.
+ "fps": 3.14, # Optional. The frame rate of the video sent to the model. If not specified, the default value will be 1.0. The fps range is (0.0, 24.0].
+ "startOffset": "A String", # Optional. The start offset of the video.
+ },
+ },
+ ],
+ "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
+ },
+ },
+ },
+ "text": "A String", # Text prompt.
+ "value": "", # Fields and values that can be used to populate the prompt template.
+ },
+ "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group.
+ "a_key": { # A group of rubrics, used for grouping rubrics based on a metric or a version.
+ "displayName": "A String", # Human-readable name for the group. This should be unique within a given context if used for display or selection. Example: "Instruction Following V1", "Content Quality - Summarization Task".
+ "groupId": "A String", # Unique identifier for the group.
+ "rubrics": [ # Rubrics that are part of this group.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ },
+ },
+ "evaluationResponse": { # Evaluation result. # Output only. The response from evaluation.
+ "candidateResults": [ # Optional. The results for the metric.
+ { # Result for a single candidate.
+ "additionalResults": "", # Optional. Additional results for the metric.
+ "candidate": "A String", # Required. The candidate that is being evaluated. The value is the same as the candidate name in the EvaluationRequest.
+ "explanation": "A String", # Optional. The explanation for the metric.
+ "metric": "A String", # Required. The metric that was evaluated.
+ "rubricVerdicts": [ # Optional. The rubric verdicts for the metric.
+ { # Represents the verdict of an evaluation against a single rubric.
+ "evaluatedRubric": { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics. # Required. The full rubric definition that was evaluated. Storing this ensures the verdict is self-contained and understandable, especially if the original rubric definition changes or was dynamically generated.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ "reasoning": "A String", # Optional. Human-readable reasoning or explanation for the verdict. This can include specific examples or details from the evaluated content that justify the given verdict.
+ "verdict": True or False, # Required. Outcome of the evaluation against the rubric, represented as a boolean. `true` indicates a "Pass", `false` indicates a "Fail".
+ },
+ ],
+ "score": 3.14, # Optional. The score for the metric.
+ },
+ ],
+ "evaluationRequest": "A String", # Required. The request item that was evaluated. Format: projects/{project}/locations/{location}/evaluationItems/{evaluation_item}
+ "evaluationRun": "A String", # Required. The evaluation run that was used to generate the result. Format: projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}
+ "metadata": "", # Optional. Metadata about the evaluation result.
+ "metric": "A String", # Required. The metric that was evaluated.
+ "request": { # Single evaluation request. # Required. The request that was evaluated.
+ "candidateResponses": [ # Optional. Responses from model under test and other baseline models for comparison.
+ { # Responses from model or agent.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ ],
+ "goldenResponse": { # Responses from model or agent. # Optional. The Ideal response or ground truth.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ "prompt": { # Prompt to be evaluated. # Required. The request/prompt to evaluate.
+ "promptTemplateData": { # Message to hold a prompt template and the values to populate the template. # Prompt template data.
+ "values": { # The values for fields in the prompt template.
+ "a_key": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
+ "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
+ { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
+ "codeExecutionResult": { # Result of executing the [ExecutableCode]. Only generated when using the [CodeExecution] tool, and always follows a `part` containing the [ExecutableCode]. # Optional. Result of executing the [ExecutableCode].
+ "outcome": "A String", # Required. Outcome of the code execution.
+ "output": "A String", # Optional. Contains stdout when code execution is successful, stderr or other description otherwise.
+ },
+ "executableCode": { # Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated. # Optional. Code generated by the model that is meant to be executed.
+ "code": "A String", # Required. The code to be executed.
+ "language": "A String", # Required. Programming language of the `code`.
+ },
+ "fileData": { # URI based data. # Optional. URI based data.
+ "displayName": "A String", # Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "fileUri": "A String", # Required. URI.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
+ "args": { # Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
+ "a_key": "", # Properties of the object.
+ },
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
+ },
+ "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
+ "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "inlineData": { # Content blob. # Optional. Inlined bytes data.
+ "data": "A String", # Required. Raw bytes.
+ "displayName": "A String", # Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "text": "A String", # Optional. Text part (can be code).
+ "thought": True or False, # Optional. Indicates if the part is thought from the model.
+ "thoughtSignature": "A String", # Optional. An opaque signature for the thought so it can be reused in subsequent requests.
+ "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
+ "endOffset": "A String", # Optional. The end offset of the video.
+ "fps": 3.14, # Optional. The frame rate of the video sent to the model. If not specified, the default value will be 1.0. The fps range is (0.0, 24.0].
+ "startOffset": "A String", # Optional. The start offset of the video.
+ },
+ },
+ ],
+ "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
+ },
+ },
+ },
+ "text": "A String", # Text prompt.
+ "value": "", # Fields and values that can be used to populate the prompt template.
+ },
+ "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group.
+ "a_key": { # A group of rubrics, used for grouping rubrics based on a metric or a version.
+ "displayName": "A String", # Human-readable name for the group. This should be unique within a given context if used for display or selection. Example: "Instruction Following V1", "Content Quality - Summarization Task".
+ "groupId": "A String", # Unique identifier for the group.
+ "rubrics": [ # Rubrics that are part of this group.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ },
+ },
+ },
+ "gcsUri": "A String", # The GCS object where the request or response is stored.
+ "labels": { # Optional. Labels for the EvaluationItem.
+ "a_key": "A String",
+ },
+ "metadata": "", # Optional. Metadata for the EvaluationItem.
+ "name": "A String", # Identifier. The resource name of the EvaluationItem. Format: `projects/{project}/locations/{location}/evaluationItems/{evaluation_item}`
+}
+
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # EvaluationItem is a single evaluation request or result. The content of an EvaluationItem is immutable - it cannot be updated once created. EvaluationItems can be deleted when no longer needed.
+ "createTime": "A String", # Output only. Timestamp when this item was created.
+ "displayName": "A String", # Required. The display name of the EvaluationItem.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Output only. Error for the evaluation item.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "evaluationItemType": "A String", # Required. The type of the EvaluationItem.
+ "evaluationRequest": { # Single evaluation request. # The request to evaluate.
+ "candidateResponses": [ # Optional. Responses from model under test and other baseline models for comparison.
+ { # Responses from model or agent.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ ],
+ "goldenResponse": { # Responses from model or agent. # Optional. The Ideal response or ground truth.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ "prompt": { # Prompt to be evaluated. # Required. The request/prompt to evaluate.
+ "promptTemplateData": { # Message to hold a prompt template and the values to populate the template. # Prompt template data.
+ "values": { # The values for fields in the prompt template.
+ "a_key": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
+ "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
+ { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
+ "codeExecutionResult": { # Result of executing the [ExecutableCode]. Only generated when using the [CodeExecution] tool, and always follows a `part` containing the [ExecutableCode]. # Optional. Result of executing the [ExecutableCode].
+ "outcome": "A String", # Required. Outcome of the code execution.
+ "output": "A String", # Optional. Contains stdout when code execution is successful, stderr or other description otherwise.
+ },
+ "executableCode": { # Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated. # Optional. Code generated by the model that is meant to be executed.
+ "code": "A String", # Required. The code to be executed.
+ "language": "A String", # Required. Programming language of the `code`.
+ },
+ "fileData": { # URI based data. # Optional. URI based data.
+ "displayName": "A String", # Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "fileUri": "A String", # Required. URI.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
+ "args": { # Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
+ "a_key": "", # Properties of the object.
+ },
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
+ },
+ "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
+ "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "inlineData": { # Content blob. # Optional. Inlined bytes data.
+ "data": "A String", # Required. Raw bytes.
+ "displayName": "A String", # Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "text": "A String", # Optional. Text part (can be code).
+ "thought": True or False, # Optional. Indicates if the part is thought from the model.
+ "thoughtSignature": "A String", # Optional. An opaque signature for the thought so it can be reused in subsequent requests.
+ "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
+ "endOffset": "A String", # Optional. The end offset of the video.
+ "fps": 3.14, # Optional. The frame rate of the video sent to the model. If not specified, the default value will be 1.0. The fps range is (0.0, 24.0].
+ "startOffset": "A String", # Optional. The start offset of the video.
+ },
+ },
+ ],
+ "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
+ },
+ },
+ },
+ "text": "A String", # Text prompt.
+ "value": "", # Fields and values that can be used to populate the prompt template.
+ },
+ "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group.
+ "a_key": { # A group of rubrics, used for grouping rubrics based on a metric or a version.
+ "displayName": "A String", # Human-readable name for the group. This should be unique within a given context if used for display or selection. Example: "Instruction Following V1", "Content Quality - Summarization Task".
+ "groupId": "A String", # Unique identifier for the group.
+ "rubrics": [ # Rubrics that are part of this group.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ },
+ },
+ "evaluationResponse": { # Evaluation result. # Output only. The response from evaluation.
+ "candidateResults": [ # Optional. The results for the metric.
+ { # Result for a single candidate.
+ "additionalResults": "", # Optional. Additional results for the metric.
+ "candidate": "A String", # Required. The candidate that is being evaluated. The value is the same as the candidate name in the EvaluationRequest.
+ "explanation": "A String", # Optional. The explanation for the metric.
+ "metric": "A String", # Required. The metric that was evaluated.
+ "rubricVerdicts": [ # Optional. The rubric verdicts for the metric.
+ { # Represents the verdict of an evaluation against a single rubric.
+ "evaluatedRubric": { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics. # Required. The full rubric definition that was evaluated. Storing this ensures the verdict is self-contained and understandable, especially if the original rubric definition changes or was dynamically generated.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ "reasoning": "A String", # Optional. Human-readable reasoning or explanation for the verdict. This can include specific examples or details from the evaluated content that justify the given verdict.
+ "verdict": True or False, # Required. Outcome of the evaluation against the rubric, represented as a boolean. `true` indicates a "Pass", `false` indicates a "Fail".
+ },
+ ],
+ "score": 3.14, # Optional. The score for the metric.
+ },
+ ],
+ "evaluationRequest": "A String", # Required. The request item that was evaluated. Format: projects/{project}/locations/{location}/evaluationItems/{evaluation_item}
+ "evaluationRun": "A String", # Required. The evaluation run that was used to generate the result. Format: projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}
+ "metadata": "", # Optional. Metadata about the evaluation result.
+ "metric": "A String", # Required. The metric that was evaluated.
+ "request": { # Single evaluation request. # Required. The request that was evaluated.
+ "candidateResponses": [ # Optional. Responses from model under test and other baseline models for comparison.
+ { # Responses from model or agent.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ ],
+ "goldenResponse": { # Responses from model or agent. # Optional. The Ideal response or ground truth.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ "prompt": { # Prompt to be evaluated. # Required. The request/prompt to evaluate.
+ "promptTemplateData": { # Message to hold a prompt template and the values to populate the template. # Prompt template data.
+ "values": { # The values for fields in the prompt template.
+ "a_key": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
+ "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
+ { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
+ "codeExecutionResult": { # Result of executing the [ExecutableCode]. Only generated when using the [CodeExecution] tool, and always follows a `part` containing the [ExecutableCode]. # Optional. Result of executing the [ExecutableCode].
+ "outcome": "A String", # Required. Outcome of the code execution.
+ "output": "A String", # Optional. Contains stdout when code execution is successful, stderr or other description otherwise.
+ },
+ "executableCode": { # Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated. # Optional. Code generated by the model that is meant to be executed.
+ "code": "A String", # Required. The code to be executed.
+ "language": "A String", # Required. Programming language of the `code`.
+ },
+ "fileData": { # URI based data. # Optional. URI based data.
+ "displayName": "A String", # Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "fileUri": "A String", # Required. URI.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
+ "args": { # Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
+ "a_key": "", # Properties of the object.
+ },
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
+ },
+ "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
+ "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "inlineData": { # Content blob. # Optional. Inlined bytes data.
+ "data": "A String", # Required. Raw bytes.
+ "displayName": "A String", # Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "text": "A String", # Optional. Text part (can be code).
+ "thought": True or False, # Optional. Indicates if the part is thought from the model.
+ "thoughtSignature": "A String", # Optional. An opaque signature for the thought so it can be reused in subsequent requests.
+ "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
+ "endOffset": "A String", # Optional. The end offset of the video.
+ "fps": 3.14, # Optional. The frame rate of the video sent to the model. If not specified, the default value will be 1.0. The fps range is (0.0, 24.0].
+ "startOffset": "A String", # Optional. The start offset of the video.
+ },
+ },
+ ],
+ "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
+ },
+ },
+ },
+ "text": "A String", # Text prompt.
+ "value": "", # Fields and values that can be used to populate the prompt template.
+ },
+ "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group.
+ "a_key": { # A group of rubrics, used for grouping rubrics based on a metric or a version.
+ "displayName": "A String", # Human-readable name for the group. This should be unique within a given context if used for display or selection. Example: "Instruction Following V1", "Content Quality - Summarization Task".
+ "groupId": "A String", # Unique identifier for the group.
+ "rubrics": [ # Rubrics that are part of this group.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ },
+ },
+ },
+ "gcsUri": "A String", # The GCS object where the request or response is stored.
+ "labels": { # Optional. Labels for the EvaluationItem.
+ "a_key": "A String",
+ },
+ "metadata": "", # Optional. Metadata for the EvaluationItem.
+ "name": "A String", # Identifier. The resource name of the EvaluationItem. Format: `projects/{project}/locations/{location}/evaluationItems/{evaluation_item}`
+}
+delete(name, x__xgafv=None)
+ Deletes an Evaluation Item.
+
+Args:
+ name: string, Required. The name of the EvaluationItem resource to be deleted. Format: `projects/{project}/locations/{location}/evaluationItems/{evaluation_item}` (required)
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # This resource represents a long-running operation that is the result of a network API call.
+ "done": True or False, # If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "metadata": { # Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any.
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the `name` should be a resource name ending with `operations/{unique_id}`.
+ "response": { # The normal, successful response of the operation. If the original method returns no data on success, such as `Delete`, the response is `google.protobuf.Empty`. If the original method is standard `Get`/`Create`/`Update`, the response should be the resource. For other methods, the response should have the type `XxxResponse`, where `Xxx` is the original method name. For example, if the original method name is `TakeSnapshot()`, the inferred response type is `TakeSnapshotResponse`.
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+}
+get(name, x__xgafv=None)
+ Gets an Evaluation Item.
+
+Args:
+ name: string, Required. The name of the EvaluationItem resource. Format: `projects/{project}/locations/{location}/evaluationItems/{evaluation_item}` (required)
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # EvaluationItem is a single evaluation request or result. The content of an EvaluationItem is immutable - it cannot be updated once created. EvaluationItems can be deleted when no longer needed.
+ "createTime": "A String", # Output only. Timestamp when this item was created.
+ "displayName": "A String", # Required. The display name of the EvaluationItem.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Output only. Error for the evaluation item.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "evaluationItemType": "A String", # Required. The type of the EvaluationItem.
+ "evaluationRequest": { # Single evaluation request. # The request to evaluate.
+ "candidateResponses": [ # Optional. Responses from model under test and other baseline models for comparison.
+ { # Responses from model or agent.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ ],
+ "goldenResponse": { # Responses from model or agent. # Optional. The Ideal response or ground truth.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ "prompt": { # Prompt to be evaluated. # Required. The request/prompt to evaluate.
+ "promptTemplateData": { # Message to hold a prompt template and the values to populate the template. # Prompt template data.
+ "values": { # The values for fields in the prompt template.
+ "a_key": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
+ "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
+ { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
+ "codeExecutionResult": { # Result of executing the [ExecutableCode]. Only generated when using the [CodeExecution] tool, and always follows a `part` containing the [ExecutableCode]. # Optional. Result of executing the [ExecutableCode].
+ "outcome": "A String", # Required. Outcome of the code execution.
+ "output": "A String", # Optional. Contains stdout when code execution is successful, stderr or other description otherwise.
+ },
+ "executableCode": { # Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated. # Optional. Code generated by the model that is meant to be executed.
+ "code": "A String", # Required. The code to be executed.
+ "language": "A String", # Required. Programming language of the `code`.
+ },
+ "fileData": { # URI based data. # Optional. URI based data.
+ "displayName": "A String", # Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "fileUri": "A String", # Required. URI.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
+ "args": { # Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
+ "a_key": "", # Properties of the object.
+ },
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
+ },
+ "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
+ "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "inlineData": { # Content blob. # Optional. Inlined bytes data.
+ "data": "A String", # Required. Raw bytes.
+ "displayName": "A String", # Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "text": "A String", # Optional. Text part (can be code).
+ "thought": True or False, # Optional. Indicates if the part is thought from the model.
+ "thoughtSignature": "A String", # Optional. An opaque signature for the thought so it can be reused in subsequent requests.
+ "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
+ "endOffset": "A String", # Optional. The end offset of the video.
+ "fps": 3.14, # Optional. The frame rate of the video sent to the model. If not specified, the default value will be 1.0. The fps range is (0.0, 24.0].
+ "startOffset": "A String", # Optional. The start offset of the video.
+ },
+ },
+ ],
+ "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
+ },
+ },
+ },
+ "text": "A String", # Text prompt.
+ "value": "", # Fields and values that can be used to populate the prompt template.
+ },
+ "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group.
+ "a_key": { # A group of rubrics, used for grouping rubrics based on a metric or a version.
+ "displayName": "A String", # Human-readable name for the group. This should be unique within a given context if used for display or selection. Example: "Instruction Following V1", "Content Quality - Summarization Task".
+ "groupId": "A String", # Unique identifier for the group.
+ "rubrics": [ # Rubrics that are part of this group.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ },
+ },
+ "evaluationResponse": { # Evaluation result. # Output only. The response from evaluation.
+ "candidateResults": [ # Optional. The results for the metric.
+ { # Result for a single candidate.
+ "additionalResults": "", # Optional. Additional results for the metric.
+ "candidate": "A String", # Required. The candidate that is being evaluated. The value is the same as the candidate name in the EvaluationRequest.
+ "explanation": "A String", # Optional. The explanation for the metric.
+ "metric": "A String", # Required. The metric that was evaluated.
+ "rubricVerdicts": [ # Optional. The rubric verdicts for the metric.
+ { # Represents the verdict of an evaluation against a single rubric.
+ "evaluatedRubric": { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics. # Required. The full rubric definition that was evaluated. Storing this ensures the verdict is self-contained and understandable, especially if the original rubric definition changes or was dynamically generated.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ "reasoning": "A String", # Optional. Human-readable reasoning or explanation for the verdict. This can include specific examples or details from the evaluated content that justify the given verdict.
+ "verdict": True or False, # Required. Outcome of the evaluation against the rubric, represented as a boolean. `true` indicates a "Pass", `false` indicates a "Fail".
+ },
+ ],
+ "score": 3.14, # Optional. The score for the metric.
+ },
+ ],
+ "evaluationRequest": "A String", # Required. The request item that was evaluated. Format: projects/{project}/locations/{location}/evaluationItems/{evaluation_item}
+ "evaluationRun": "A String", # Required. The evaluation run that was used to generate the result. Format: projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}
+ "metadata": "", # Optional. Metadata about the evaluation result.
+ "metric": "A String", # Required. The metric that was evaluated.
+ "request": { # Single evaluation request. # Required. The request that was evaluated.
+ "candidateResponses": [ # Optional. Responses from model under test and other baseline models for comparison.
+ { # Responses from model or agent.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ ],
+ "goldenResponse": { # Responses from model or agent. # Optional. The Ideal response or ground truth.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ "prompt": { # Prompt to be evaluated. # Required. The request/prompt to evaluate.
+ "promptTemplateData": { # Message to hold a prompt template and the values to populate the template. # Prompt template data.
+ "values": { # The values for fields in the prompt template.
+ "a_key": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
+ "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
+ { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
+ "codeExecutionResult": { # Result of executing the [ExecutableCode]. Only generated when using the [CodeExecution] tool, and always follows a `part` containing the [ExecutableCode]. # Optional. Result of executing the [ExecutableCode].
+ "outcome": "A String", # Required. Outcome of the code execution.
+ "output": "A String", # Optional. Contains stdout when code execution is successful, stderr or other description otherwise.
+ },
+ "executableCode": { # Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated. # Optional. Code generated by the model that is meant to be executed.
+ "code": "A String", # Required. The code to be executed.
+ "language": "A String", # Required. Programming language of the `code`.
+ },
+ "fileData": { # URI based data. # Optional. URI based data.
+ "displayName": "A String", # Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "fileUri": "A String", # Required. URI.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
+ "args": { # Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
+ "a_key": "", # Properties of the object.
+ },
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
+ },
+ "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
+ "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "inlineData": { # Content blob. # Optional. Inlined bytes data.
+ "data": "A String", # Required. Raw bytes.
+ "displayName": "A String", # Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "text": "A String", # Optional. Text part (can be code).
+ "thought": True or False, # Optional. Indicates if the part is thought from the model.
+ "thoughtSignature": "A String", # Optional. An opaque signature for the thought so it can be reused in subsequent requests.
+ "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
+ "endOffset": "A String", # Optional. The end offset of the video.
+ "fps": 3.14, # Optional. The frame rate of the video sent to the model. If not specified, the default value will be 1.0. The fps range is (0.0, 24.0].
+ "startOffset": "A String", # Optional. The start offset of the video.
+ },
+ },
+ ],
+ "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
+ },
+ },
+ },
+ "text": "A String", # Text prompt.
+ "value": "", # Fields and values that can be used to populate the prompt template.
+ },
+ "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group.
+ "a_key": { # A group of rubrics, used for grouping rubrics based on a metric or a version.
+ "displayName": "A String", # Human-readable name for the group. This should be unique within a given context if used for display or selection. Example: "Instruction Following V1", "Content Quality - Summarization Task".
+ "groupId": "A String", # Unique identifier for the group.
+ "rubrics": [ # Rubrics that are part of this group.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ },
+ },
+ },
+ "gcsUri": "A String", # The GCS object where the request or response is stored.
+ "labels": { # Optional. Labels for the EvaluationItem.
+ "a_key": "A String",
+ },
+ "metadata": "", # Optional. Metadata for the EvaluationItem.
+ "name": "A String", # Identifier. The resource name of the EvaluationItem. Format: `projects/{project}/locations/{location}/evaluationItems/{evaluation_item}`
+}
+list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
+ Lists Evaluation Items.
+
+Args:
+ parent: string, Required. The resource name of the Location from which to list the Evaluation Items. Format: `projects/{project}/locations/{location}` (required)
+ filter: string, Optional. Filter expression that matches a subset of the EvaluationItems to show. For field names both snake_case and camelCase are supported. For more information about filter syntax, see [AIP-160](https://google.aip.dev/160).
+ orderBy: string, Optional. A comma-separated list of fields to order by, sorted in ascending order by default. Use `desc` after a field name for descending.
+ pageSize: integer, Optional. The maximum number of Evaluation Items to return.
+ pageToken: string, Optional. A page token, received from a previous `ListEvaluationItems` call. Provide this to retrieve the subsequent page.
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # Response message for EvaluationManagementService.ListEvaluationItems.
+ "evaluationItems": [ # List of EvaluationItems in the requested page.
+ { # EvaluationItem is a single evaluation request or result. The content of an EvaluationItem is immutable - it cannot be updated once created. EvaluationItems can be deleted when no longer needed.
+ "createTime": "A String", # Output only. Timestamp when this item was created.
+ "displayName": "A String", # Required. The display name of the EvaluationItem.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Output only. Error for the evaluation item.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "evaluationItemType": "A String", # Required. The type of the EvaluationItem.
+ "evaluationRequest": { # Single evaluation request. # The request to evaluate.
+ "candidateResponses": [ # Optional. Responses from model under test and other baseline models for comparison.
+ { # Responses from model or agent.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ ],
+ "goldenResponse": { # Responses from model or agent. # Optional. The Ideal response or ground truth.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ "prompt": { # Prompt to be evaluated. # Required. The request/prompt to evaluate.
+ "promptTemplateData": { # Message to hold a prompt template and the values to populate the template. # Prompt template data.
+ "values": { # The values for fields in the prompt template.
+ "a_key": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
+ "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
+ { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
+ "codeExecutionResult": { # Result of executing the [ExecutableCode]. Only generated when using the [CodeExecution] tool, and always follows a `part` containing the [ExecutableCode]. # Optional. Result of executing the [ExecutableCode].
+ "outcome": "A String", # Required. Outcome of the code execution.
+ "output": "A String", # Optional. Contains stdout when code execution is successful, stderr or other description otherwise.
+ },
+ "executableCode": { # Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated. # Optional. Code generated by the model that is meant to be executed.
+ "code": "A String", # Required. The code to be executed.
+ "language": "A String", # Required. Programming language of the `code`.
+ },
+ "fileData": { # URI based data. # Optional. URI based data.
+ "displayName": "A String", # Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "fileUri": "A String", # Required. URI.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
+ "args": { # Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
+ "a_key": "", # Properties of the object.
+ },
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
+ },
+ "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
+ "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "inlineData": { # Content blob. # Optional. Inlined bytes data.
+ "data": "A String", # Required. Raw bytes.
+ "displayName": "A String", # Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "text": "A String", # Optional. Text part (can be code).
+ "thought": True or False, # Optional. Indicates if the part is thought from the model.
+ "thoughtSignature": "A String", # Optional. An opaque signature for the thought so it can be reused in subsequent requests.
+ "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
+ "endOffset": "A String", # Optional. The end offset of the video.
+ "fps": 3.14, # Optional. The frame rate of the video sent to the model. If not specified, the default value will be 1.0. The fps range is (0.0, 24.0].
+ "startOffset": "A String", # Optional. The start offset of the video.
+ },
+ },
+ ],
+ "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
+ },
+ },
+ },
+ "text": "A String", # Text prompt.
+ "value": "", # Fields and values that can be used to populate the prompt template.
+ },
+ "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group.
+ "a_key": { # A group of rubrics, used for grouping rubrics based on a metric or a version.
+ "displayName": "A String", # Human-readable name for the group. This should be unique within a given context if used for display or selection. Example: "Instruction Following V1", "Content Quality - Summarization Task".
+ "groupId": "A String", # Unique identifier for the group.
+ "rubrics": [ # Rubrics that are part of this group.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ },
+ },
+ "evaluationResponse": { # Evaluation result. # Output only. The response from evaluation.
+ "candidateResults": [ # Optional. The results for the metric.
+ { # Result for a single candidate.
+ "additionalResults": "", # Optional. Additional results for the metric.
+ "candidate": "A String", # Required. The candidate that is being evaluated. The value is the same as the candidate name in the EvaluationRequest.
+ "explanation": "A String", # Optional. The explanation for the metric.
+ "metric": "A String", # Required. The metric that was evaluated.
+ "rubricVerdicts": [ # Optional. The rubric verdicts for the metric.
+ { # Represents the verdict of an evaluation against a single rubric.
+ "evaluatedRubric": { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics. # Required. The full rubric definition that was evaluated. Storing this ensures the verdict is self-contained and understandable, especially if the original rubric definition changes or was dynamically generated.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ "reasoning": "A String", # Optional. Human-readable reasoning or explanation for the verdict. This can include specific examples or details from the evaluated content that justify the given verdict.
+ "verdict": True or False, # Required. Outcome of the evaluation against the rubric, represented as a boolean. `true` indicates a "Pass", `false` indicates a "Fail".
+ },
+ ],
+ "score": 3.14, # Optional. The score for the metric.
+ },
+ ],
+ "evaluationRequest": "A String", # Required. The request item that was evaluated. Format: projects/{project}/locations/{location}/evaluationItems/{evaluation_item}
+ "evaluationRun": "A String", # Required. The evaluation run that was used to generate the result. Format: projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}
+ "metadata": "", # Optional. Metadata about the evaluation result.
+ "metric": "A String", # Required. The metric that was evaluated.
+ "request": { # Single evaluation request. # Required. The request that was evaluated.
+ "candidateResponses": [ # Optional. Responses from model under test and other baseline models for comparison.
+ { # Responses from model or agent.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ ],
+ "goldenResponse": { # Responses from model or agent. # Optional. The Ideal response or ground truth.
+ "candidate": "A String", # Required. The name of the candidate that produced the response.
+ "text": "A String", # Text response.
+ "value": "", # Fields and values that can be used to populate the response template.
+ },
+ "prompt": { # Prompt to be evaluated. # Required. The request/prompt to evaluate.
+ "promptTemplateData": { # Message to hold a prompt template and the values to populate the template. # Prompt template data.
+ "values": { # The values for fields in the prompt template.
+ "a_key": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
+ "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
+ { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
+ "codeExecutionResult": { # Result of executing the [ExecutableCode]. Only generated when using the [CodeExecution] tool, and always follows a `part` containing the [ExecutableCode]. # Optional. Result of executing the [ExecutableCode].
+ "outcome": "A String", # Required. Outcome of the code execution.
+ "output": "A String", # Optional. Contains stdout when code execution is successful, stderr or other description otherwise.
+ },
+ "executableCode": { # Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated. # Optional. Code generated by the model that is meant to be executed.
+ "code": "A String", # Required. The code to be executed.
+ "language": "A String", # Required. Programming language of the `code`.
+ },
+ "fileData": { # URI based data. # Optional. URI based data.
+ "displayName": "A String", # Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "fileUri": "A String", # Required. URI.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
+ "args": { # Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
+ "a_key": "", # Properties of the object.
+ },
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
+ },
+ "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
+ "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "inlineData": { # Content blob. # Optional. Inlined bytes data.
+ "data": "A String", # Required. Raw bytes.
+ "displayName": "A String", # Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "text": "A String", # Optional. Text part (can be code).
+ "thought": True or False, # Optional. Indicates if the part is thought from the model.
+ "thoughtSignature": "A String", # Optional. An opaque signature for the thought so it can be reused in subsequent requests.
+ "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
+ "endOffset": "A String", # Optional. The end offset of the video.
+ "fps": 3.14, # Optional. The frame rate of the video sent to the model. If not specified, the default value will be 1.0. The fps range is (0.0, 24.0].
+ "startOffset": "A String", # Optional. The start offset of the video.
+ },
+ },
+ ],
+ "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
+ },
+ },
+ },
+ "text": "A String", # Text prompt.
+ "value": "", # Fields and values that can be used to populate the prompt template.
+ },
+ "rubrics": { # Optional. Named groups of rubrics associated with this prompt. The key is a user-defined name for the rubric group.
+ "a_key": { # A group of rubrics, used for grouping rubrics based on a metric or a version.
+ "displayName": "A String", # Human-readable name for the group. This should be unique within a given context if used for display or selection. Example: "Instruction Following V1", "Content Quality - Summarization Task".
+ "groupId": "A String", # Unique identifier for the group.
+ "rubrics": [ # Rubrics that are part of this group.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ },
+ },
+ },
+ "gcsUri": "A String", # The GCS object where the request or response is stored.
+ "labels": { # Optional. Labels for the EvaluationItem.
+ "a_key": "A String",
+ },
+ "metadata": "", # Optional. Metadata for the EvaluationItem.
+ "name": "A String", # Identifier. The resource name of the EvaluationItem. Format: `projects/{project}/locations/{location}/evaluationItems/{evaluation_item}`
+ },
+ ],
+ "nextPageToken": "A String", # A token to retrieve the next page of results.
+}
+list_next()
+ Retrieves the next page of results. + + Args: + previous_request: The request for the previous page. (required) + previous_response: The response from the request for the previous page. (required) + + Returns: + A request object that you can call 'execute()' on to request the next + page. Returns None if there are no more items in the collection. ++
+ cancel(name, body=None, x__xgafv=None)
Cancels an Evaluation Run. Attempts to cancel a running Evaluation Run asynchronously. Status of run can be checked via GetEvaluationRun.
+
+ close()
Close httplib2 connections.
+
+ create(parent, body=None, x__xgafv=None)
Creates an Evaluation Run.
+ +Deletes an Evaluation Run.
+ +Gets an Evaluation Run.
+
+ list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
Lists Evaluation Runs.
+ +Retrieves the next page of results.
+cancel(name, body=None, x__xgafv=None)
+ Cancels an Evaluation Run. Attempts to cancel a running Evaluation Run asynchronously. Status of run can be checked via GetEvaluationRun.
+
+Args:
+ name: string, Required. The name of the EvaluationRun resource to be cancelled. Format: `projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}` (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # Request message for EvaluationManagementService.CancelEvaluationRun.
+}
+
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); }
+}
+close()
+ Close httplib2 connections.+
create(parent, body=None, x__xgafv=None)
+ Creates an Evaluation Run.
+
+Args:
+ parent: string, Required. The resource name of the Location to create the Evaluation Run in. Format: `projects/{project}/locations/{location}` (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # EvaluationRun is a resource that represents a single evaluation run, which includes a set of prompts, model responses, evaluation configuration and the resulting metrics.
+ "completionTime": "A String", # Output only. Time when the evaluation run was completed.
+ "createTime": "A String", # Output only. Time when the evaluation run was created.
+ "dataSource": { # The data source for the evaluation run. # Required. The data source for the evaluation run.
+ "bigqueryRequestSet": { # The request set for the evaluation run. # Evaluation data in bigquery.
+ "candidateResponseColumns": { # Optional. Map of candidate name to candidate response column name. The column will be in evaluation_item.CandidateResponse format.
+ "a_key": "A String",
+ },
+ "promptColumn": "A String", # Optional. The name of the column that contains the requests to evaluate. This will be in evaluation_item.EvalPrompt format.
+ "rubricsColumn": "A String", # Optional. The name of the column that contains the rubrics. This will be in evaluation_rubric.RubricGroup format (cl/762595858).
+ "samplingConfig": { # The sampling config. # Optional. The sampling config for the bigquery resource.
+ "samplingCount": 42, # Optional. The total number of logged data to import. If available data is less than the sampling count, all data will be imported. Default is 100.
+ "samplingDuration": "A String", # Optional. How long to wait before sampling data from the BigQuery table. If not specified, defaults to 0.
+ "samplingMethod": "A String", # Optional. The sampling method to use.
+ },
+ "uri": "A String", # Required. The URI of a BigQuery table. e.g. bq://projectId.bqDatasetId.bqTableId
+ },
+ "evaluationSet": "A String", # The EvaluationSet resource name. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}`
+ },
+ "displayName": "A String", # Required. The display name of the Evaluation Run.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Output only. Only populated when the evaluation run's state is FAILED or CANCELLED.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "evaluationConfig": { # The Evalution configuration used for the evaluation run. # Required. The configuration used for the evaluation.
+ "autoraterConfig": { # The autorater config used for the evaluation run. # Optional. The autorater config for the evaluation run.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metrics": [ # Required. The metrics to be calculated in the evaluation run.
+ { # The metric used for evaluation runs.
+ "computationBasedMetricSpec": { # Specification for a computation based metric. # Spec for a computation based metric.
+ "parameters": { # Optional. A map of parameters for the metric, e.g. {"rouge_type": "rougeL"}.
+ "a_key": "", # Properties of the object.
+ },
+ "type": "A String", # Required. The type of the computation based metric.
+ },
+ "llmBasedMetricSpec": { # Specification for an LLM based metric. # Spec for an LLM based metric.
+ "additionalConfig": { # Optional. Optional additional configuration for the metric.
+ "a_key": "", # Properties of the object.
+ },
+ "judgeAutoraterConfig": { # The autorater config used for the evaluation run. # Optional. Optional configuration for the judge LLM (Autorater).
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metricPromptTemplate": "A String", # Required. Template for the prompt sent to the judge model.
+ "predefinedRubricGenerationSpec": { # Specification for a pre-defined metric. # Dynamically generate rubrics using a predefined spec.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance.
+ "systemInstruction": "A String", # Optional. System instructions for the judge model.
+ },
+ "metric": "A String", # Required. The name of the metric.
+ "predefinedMetricSpec": { # Specification for a pre-defined metric. # Spec for a pre-defined metric.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricBasedMetricSpec": { # Specification for a metric that is based on rubrics. # Spec for rubric based metric.
+ "inlineRubrics": { # Defines a list of rubrics, used when providing rubrics inline. # Use rubrics provided directly in the spec.
+ "rubrics": [ # The list of rubrics.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ "judgeAutoraterConfig": { # The autorater config used for the evaluation run. # Optional. Optional configuration for the judge LLM (Autorater). The definition of AutoraterConfig needs to be provided.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metricPromptTemplate": "A String", # Optional. Template for the prompt used by the judge model to evaluate against rubrics.
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics for evaluation using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input content. This refers to a key in the `rubric_groups` map of `RubricEnhancedContents`.
+ },
+ },
+ ],
+ "outputConfig": { # The output config for the evaluation run. # Optional. The output config for the evaluation run.
+ "bigqueryDestination": { # The BigQuery location for the output content. # BigQuery destination for evaluation output.
+ "outputUri": "A String", # Required. BigQuery URI to a project or table, up to 2000 characters long. When only the project is specified, the Dataset and Table is created. When the full table reference is specified, the Dataset must exist and table must not exist. Accepted forms: * BigQuery path. For example: `bq://projectId` or `bq://projectId.bqDatasetId` or `bq://projectId.bqDatasetId.bqTableId`.
+ },
+ "gcsDestination": { # The Google Cloud Storage location where the output is to be written to. # Cloud Storage destination for evaluation output.
+ "outputUriPrefix": "A String", # Required. Google Cloud Storage URI to output directory. If the uri doesn't end with '/', a '/' will be automatically appended. The directory is created if it doesn't exist.
+ },
+ },
+ "promptTemplate": { # Prompt template used for inference. # The prompt template used for inference. The values for variables in the prompt template are defined in EvaluationItem.EvaluationPrompt.PromptTemplateData.values.
+ "gcsUri": "A String", # Prompt template stored in Cloud Storage. Format: "gs://my-bucket/file-name.txt".
+ "promptTemplate": "A String", # Inline prompt template. Template variables should be in the format "{var_name}". Example: "Translate the following from {source_lang} to {target_lang}: {text}"
+ },
+ "rubricConfigs": [ # Optional. The rubric configs for the evaluation run. They are used to generate rubrics which can be used by rubric-based metrics. Multiple rubric configs can be specified for rubric generation but only one rubric config can be used for a rubric-based metric. If more than one rubric config is provided, the evaluation metric must specify a rubric group key. Note that if a generation spec is specified on both a rubric config and an evaluation metric, the rubrics generated for the metric will be used for evaluation.
+ { # Configuration for a rubric group to be generated/saved for evaluation.
+ "predefinedRubricGenerationSpec": { # Specification for a pre-defined metric. # Dynamically generate rubrics using a predefined spec.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Required. The key used to save the generated rubrics. If a generation spec is provided, this key will be used for the name of the generated rubric group. Otherwise, this key will be used to look up the existing rubric group on the evaluation item. Note that if a rubric group key is specified on both a rubric config and an evaluation metric, the key from the metric will be used to select the rubrics for evaluation.
+ },
+ ],
+ },
+ "evaluationResults": { # The results of the evaluation run. # Output only. The results of the evaluation run. Only populated when the evaluation run's state is SUCCEEDED.
+ "evaluationSet": "A String", # The evaluation set where item level results are stored.
+ "summaryMetrics": { # The summary metrics for the evaluation run. # Optional. The summary metrics for the evaluation run.
+ "failedItems": 42, # Optional. The number of items that failed to be evaluated.
+ "metrics": { # Optional. Map of metric name to metric value.
+ "a_key": "",
+ },
+ "totalItems": 42, # Optional. The total number of items that were evaluated.
+ },
+ },
+ "evaluationSetSnapshot": "A String", # Output only. The specific evaluation set of the evaluation run. For runs with an evaluation set input, this will be that same set. For runs with BigQuery input, it's the sampled BigQuery dataset.
+ "inferenceConfigs": { # Optional. The candidate to inference config map for the evaluation run. The candidate can be up to 128 characters long and can consist of any UTF-8 characters.
+ "a_key": { # An inference config used for model inference during the evaluation run.
+ "generationConfig": { # Generation config. # Optional. Generation config.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "model": "A String", # Required. The fully qualified name of the publisher model or endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ },
+ },
+ "labels": { # Optional. Labels for the evaluation run.
+ "a_key": "A String",
+ },
+ "metadata": "", # Optional. Metadata about the evaluation run, can be used by the caller to store additional tracking information about the evaluation run.
+ "name": "A String", # Identifier. The resource name of the EvaluationRun. This is a unique identifier. Format: `projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}`
+ "state": "A String", # Output only. The state of the evaluation run.
+}
+
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # EvaluationRun is a resource that represents a single evaluation run, which includes a set of prompts, model responses, evaluation configuration and the resulting metrics.
+ "completionTime": "A String", # Output only. Time when the evaluation run was completed.
+ "createTime": "A String", # Output only. Time when the evaluation run was created.
+ "dataSource": { # The data source for the evaluation run. # Required. The data source for the evaluation run.
+ "bigqueryRequestSet": { # The request set for the evaluation run. # Evaluation data in bigquery.
+ "candidateResponseColumns": { # Optional. Map of candidate name to candidate response column name. The column will be in evaluation_item.CandidateResponse format.
+ "a_key": "A String",
+ },
+ "promptColumn": "A String", # Optional. The name of the column that contains the requests to evaluate. This will be in evaluation_item.EvalPrompt format.
+ "rubricsColumn": "A String", # Optional. The name of the column that contains the rubrics. This will be in evaluation_rubric.RubricGroup format (cl/762595858).
+ "samplingConfig": { # The sampling config. # Optional. The sampling config for the bigquery resource.
+ "samplingCount": 42, # Optional. The total number of logged data to import. If available data is less than the sampling count, all data will be imported. Default is 100.
+ "samplingDuration": "A String", # Optional. How long to wait before sampling data from the BigQuery table. If not specified, defaults to 0.
+ "samplingMethod": "A String", # Optional. The sampling method to use.
+ },
+ "uri": "A String", # Required. The URI of a BigQuery table. e.g. bq://projectId.bqDatasetId.bqTableId
+ },
+ "evaluationSet": "A String", # The EvaluationSet resource name. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}`
+ },
+ "displayName": "A String", # Required. The display name of the Evaluation Run.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Output only. Only populated when the evaluation run's state is FAILED or CANCELLED.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "evaluationConfig": { # The Evalution configuration used for the evaluation run. # Required. The configuration used for the evaluation.
+ "autoraterConfig": { # The autorater config used for the evaluation run. # Optional. The autorater config for the evaluation run.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metrics": [ # Required. The metrics to be calculated in the evaluation run.
+ { # The metric used for evaluation runs.
+ "computationBasedMetricSpec": { # Specification for a computation based metric. # Spec for a computation based metric.
+ "parameters": { # Optional. A map of parameters for the metric, e.g. {"rouge_type": "rougeL"}.
+ "a_key": "", # Properties of the object.
+ },
+ "type": "A String", # Required. The type of the computation based metric.
+ },
+ "llmBasedMetricSpec": { # Specification for an LLM based metric. # Spec for an LLM based metric.
+ "additionalConfig": { # Optional. Optional additional configuration for the metric.
+ "a_key": "", # Properties of the object.
+ },
+ "judgeAutoraterConfig": { # The autorater config used for the evaluation run. # Optional. Optional configuration for the judge LLM (Autorater).
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metricPromptTemplate": "A String", # Required. Template for the prompt sent to the judge model.
+ "predefinedRubricGenerationSpec": { # Specification for a pre-defined metric. # Dynamically generate rubrics using a predefined spec.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance.
+ "systemInstruction": "A String", # Optional. System instructions for the judge model.
+ },
+ "metric": "A String", # Required. The name of the metric.
+ "predefinedMetricSpec": { # Specification for a pre-defined metric. # Spec for a pre-defined metric.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricBasedMetricSpec": { # Specification for a metric that is based on rubrics. # Spec for rubric based metric.
+ "inlineRubrics": { # Defines a list of rubrics, used when providing rubrics inline. # Use rubrics provided directly in the spec.
+ "rubrics": [ # The list of rubrics.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ "judgeAutoraterConfig": { # The autorater config used for the evaluation run. # Optional. Optional configuration for the judge LLM (Autorater). The definition of AutoraterConfig needs to be provided.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metricPromptTemplate": "A String", # Optional. Template for the prompt used by the judge model to evaluate against rubrics.
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics for evaluation using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input content. This refers to a key in the `rubric_groups` map of `RubricEnhancedContents`.
+ },
+ },
+ ],
+ "outputConfig": { # The output config for the evaluation run. # Optional. The output config for the evaluation run.
+ "bigqueryDestination": { # The BigQuery location for the output content. # BigQuery destination for evaluation output.
+ "outputUri": "A String", # Required. BigQuery URI to a project or table, up to 2000 characters long. When only the project is specified, the Dataset and Table is created. When the full table reference is specified, the Dataset must exist and table must not exist. Accepted forms: * BigQuery path. For example: `bq://projectId` or `bq://projectId.bqDatasetId` or `bq://projectId.bqDatasetId.bqTableId`.
+ },
+ "gcsDestination": { # The Google Cloud Storage location where the output is to be written to. # Cloud Storage destination for evaluation output.
+ "outputUriPrefix": "A String", # Required. Google Cloud Storage URI to output directory. If the uri doesn't end with '/', a '/' will be automatically appended. The directory is created if it doesn't exist.
+ },
+ },
+ "promptTemplate": { # Prompt template used for inference. # The prompt template used for inference. The values for variables in the prompt template are defined in EvaluationItem.EvaluationPrompt.PromptTemplateData.values.
+ "gcsUri": "A String", # Prompt template stored in Cloud Storage. Format: "gs://my-bucket/file-name.txt".
+ "promptTemplate": "A String", # Inline prompt template. Template variables should be in the format "{var_name}". Example: "Translate the following from {source_lang} to {target_lang}: {text}"
+ },
+ "rubricConfigs": [ # Optional. The rubric configs for the evaluation run. They are used to generate rubrics which can be used by rubric-based metrics. Multiple rubric configs can be specified for rubric generation but only one rubric config can be used for a rubric-based metric. If more than one rubric config is provided, the evaluation metric must specify a rubric group key. Note that if a generation spec is specified on both a rubric config and an evaluation metric, the rubrics generated for the metric will be used for evaluation.
+ { # Configuration for a rubric group to be generated/saved for evaluation.
+ "predefinedRubricGenerationSpec": { # Specification for a pre-defined metric. # Dynamically generate rubrics using a predefined spec.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Required. The key used to save the generated rubrics. If a generation spec is provided, this key will be used for the name of the generated rubric group. Otherwise, this key will be used to look up the existing rubric group on the evaluation item. Note that if a rubric group key is specified on both a rubric config and an evaluation metric, the key from the metric will be used to select the rubrics for evaluation.
+ },
+ ],
+ },
+ "evaluationResults": { # The results of the evaluation run. # Output only. The results of the evaluation run. Only populated when the evaluation run's state is SUCCEEDED.
+ "evaluationSet": "A String", # The evaluation set where item level results are stored.
+ "summaryMetrics": { # The summary metrics for the evaluation run. # Optional. The summary metrics for the evaluation run.
+ "failedItems": 42, # Optional. The number of items that failed to be evaluated.
+ "metrics": { # Optional. Map of metric name to metric value.
+ "a_key": "",
+ },
+ "totalItems": 42, # Optional. The total number of items that were evaluated.
+ },
+ },
+ "evaluationSetSnapshot": "A String", # Output only. The specific evaluation set of the evaluation run. For runs with an evaluation set input, this will be that same set. For runs with BigQuery input, it's the sampled BigQuery dataset.
+ "inferenceConfigs": { # Optional. The candidate to inference config map for the evaluation run. The candidate can be up to 128 characters long and can consist of any UTF-8 characters.
+ "a_key": { # An inference config used for model inference during the evaluation run.
+ "generationConfig": { # Generation config. # Optional. Generation config.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "model": "A String", # Required. The fully qualified name of the publisher model or endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ },
+ },
+ "labels": { # Optional. Labels for the evaluation run.
+ "a_key": "A String",
+ },
+ "metadata": "", # Optional. Metadata about the evaluation run, can be used by the caller to store additional tracking information about the evaluation run.
+ "name": "A String", # Identifier. The resource name of the EvaluationRun. This is a unique identifier. Format: `projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}`
+ "state": "A String", # Output only. The state of the evaluation run.
+}
+delete(name, x__xgafv=None)
+ Deletes an Evaluation Run.
+
+Args:
+ name: string, Required. The name of the EvaluationRun resource to be deleted. Format: `projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}` (required)
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # This resource represents a long-running operation that is the result of a network API call.
+ "done": True or False, # If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "metadata": { # Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any.
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the `name` should be a resource name ending with `operations/{unique_id}`.
+ "response": { # The normal, successful response of the operation. If the original method returns no data on success, such as `Delete`, the response is `google.protobuf.Empty`. If the original method is standard `Get`/`Create`/`Update`, the response should be the resource. For other methods, the response should have the type `XxxResponse`, where `Xxx` is the original method name. For example, if the original method name is `TakeSnapshot()`, the inferred response type is `TakeSnapshotResponse`.
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+}
+get(name, x__xgafv=None)
+ Gets an Evaluation Run.
+
+Args:
+ name: string, Required. The name of the EvaluationRun resource. Format: `projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}` (required)
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # EvaluationRun is a resource that represents a single evaluation run, which includes a set of prompts, model responses, evaluation configuration and the resulting metrics.
+ "completionTime": "A String", # Output only. Time when the evaluation run was completed.
+ "createTime": "A String", # Output only. Time when the evaluation run was created.
+ "dataSource": { # The data source for the evaluation run. # Required. The data source for the evaluation run.
+ "bigqueryRequestSet": { # The request set for the evaluation run. # Evaluation data in bigquery.
+ "candidateResponseColumns": { # Optional. Map of candidate name to candidate response column name. The column will be in evaluation_item.CandidateResponse format.
+ "a_key": "A String",
+ },
+ "promptColumn": "A String", # Optional. The name of the column that contains the requests to evaluate. This will be in evaluation_item.EvalPrompt format.
+ "rubricsColumn": "A String", # Optional. The name of the column that contains the rubrics. This will be in evaluation_rubric.RubricGroup format (cl/762595858).
+ "samplingConfig": { # The sampling config. # Optional. The sampling config for the bigquery resource.
+ "samplingCount": 42, # Optional. The total number of logged data to import. If available data is less than the sampling count, all data will be imported. Default is 100.
+ "samplingDuration": "A String", # Optional. How long to wait before sampling data from the BigQuery table. If not specified, defaults to 0.
+ "samplingMethod": "A String", # Optional. The sampling method to use.
+ },
+ "uri": "A String", # Required. The URI of a BigQuery table. e.g. bq://projectId.bqDatasetId.bqTableId
+ },
+ "evaluationSet": "A String", # The EvaluationSet resource name. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}`
+ },
+ "displayName": "A String", # Required. The display name of the Evaluation Run.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Output only. Only populated when the evaluation run's state is FAILED or CANCELLED.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "evaluationConfig": { # The Evalution configuration used for the evaluation run. # Required. The configuration used for the evaluation.
+ "autoraterConfig": { # The autorater config used for the evaluation run. # Optional. The autorater config for the evaluation run.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metrics": [ # Required. The metrics to be calculated in the evaluation run.
+ { # The metric used for evaluation runs.
+ "computationBasedMetricSpec": { # Specification for a computation based metric. # Spec for a computation based metric.
+ "parameters": { # Optional. A map of parameters for the metric, e.g. {"rouge_type": "rougeL"}.
+ "a_key": "", # Properties of the object.
+ },
+ "type": "A String", # Required. The type of the computation based metric.
+ },
+ "llmBasedMetricSpec": { # Specification for an LLM based metric. # Spec for an LLM based metric.
+ "additionalConfig": { # Optional. Optional additional configuration for the metric.
+ "a_key": "", # Properties of the object.
+ },
+ "judgeAutoraterConfig": { # The autorater config used for the evaluation run. # Optional. Optional configuration for the judge LLM (Autorater).
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metricPromptTemplate": "A String", # Required. Template for the prompt sent to the judge model.
+ "predefinedRubricGenerationSpec": { # Specification for a pre-defined metric. # Dynamically generate rubrics using a predefined spec.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance.
+ "systemInstruction": "A String", # Optional. System instructions for the judge model.
+ },
+ "metric": "A String", # Required. The name of the metric.
+ "predefinedMetricSpec": { # Specification for a pre-defined metric. # Spec for a pre-defined metric.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricBasedMetricSpec": { # Specification for a metric that is based on rubrics. # Spec for rubric based metric.
+ "inlineRubrics": { # Defines a list of rubrics, used when providing rubrics inline. # Use rubrics provided directly in the spec.
+ "rubrics": [ # The list of rubrics.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ "judgeAutoraterConfig": { # The autorater config used for the evaluation run. # Optional. Optional configuration for the judge LLM (Autorater). The definition of AutoraterConfig needs to be provided.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metricPromptTemplate": "A String", # Optional. Template for the prompt used by the judge model to evaluate against rubrics.
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics for evaluation using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input content. This refers to a key in the `rubric_groups` map of `RubricEnhancedContents`.
+ },
+ },
+ ],
+ "outputConfig": { # The output config for the evaluation run. # Optional. The output config for the evaluation run.
+ "bigqueryDestination": { # The BigQuery location for the output content. # BigQuery destination for evaluation output.
+ "outputUri": "A String", # Required. BigQuery URI to a project or table, up to 2000 characters long. When only the project is specified, the Dataset and Table is created. When the full table reference is specified, the Dataset must exist and table must not exist. Accepted forms: * BigQuery path. For example: `bq://projectId` or `bq://projectId.bqDatasetId` or `bq://projectId.bqDatasetId.bqTableId`.
+ },
+ "gcsDestination": { # The Google Cloud Storage location where the output is to be written to. # Cloud Storage destination for evaluation output.
+ "outputUriPrefix": "A String", # Required. Google Cloud Storage URI to output directory. If the uri doesn't end with '/', a '/' will be automatically appended. The directory is created if it doesn't exist.
+ },
+ },
+ "promptTemplate": { # Prompt template used for inference. # The prompt template used for inference. The values for variables in the prompt template are defined in EvaluationItem.EvaluationPrompt.PromptTemplateData.values.
+ "gcsUri": "A String", # Prompt template stored in Cloud Storage. Format: "gs://my-bucket/file-name.txt".
+ "promptTemplate": "A String", # Inline prompt template. Template variables should be in the format "{var_name}". Example: "Translate the following from {source_lang} to {target_lang}: {text}"
+ },
+ "rubricConfigs": [ # Optional. The rubric configs for the evaluation run. They are used to generate rubrics which can be used by rubric-based metrics. Multiple rubric configs can be specified for rubric generation but only one rubric config can be used for a rubric-based metric. If more than one rubric config is provided, the evaluation metric must specify a rubric group key. Note that if a generation spec is specified on both a rubric config and an evaluation metric, the rubrics generated for the metric will be used for evaluation.
+ { # Configuration for a rubric group to be generated/saved for evaluation.
+ "predefinedRubricGenerationSpec": { # Specification for a pre-defined metric. # Dynamically generate rubrics using a predefined spec.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Required. The key used to save the generated rubrics. If a generation spec is provided, this key will be used for the name of the generated rubric group. Otherwise, this key will be used to look up the existing rubric group on the evaluation item. Note that if a rubric group key is specified on both a rubric config and an evaluation metric, the key from the metric will be used to select the rubrics for evaluation.
+ },
+ ],
+ },
+ "evaluationResults": { # The results of the evaluation run. # Output only. The results of the evaluation run. Only populated when the evaluation run's state is SUCCEEDED.
+ "evaluationSet": "A String", # The evaluation set where item level results are stored.
+ "summaryMetrics": { # The summary metrics for the evaluation run. # Optional. The summary metrics for the evaluation run.
+ "failedItems": 42, # Optional. The number of items that failed to be evaluated.
+ "metrics": { # Optional. Map of metric name to metric value.
+ "a_key": "",
+ },
+ "totalItems": 42, # Optional. The total number of items that were evaluated.
+ },
+ },
+ "evaluationSetSnapshot": "A String", # Output only. The specific evaluation set of the evaluation run. For runs with an evaluation set input, this will be that same set. For runs with BigQuery input, it's the sampled BigQuery dataset.
+ "inferenceConfigs": { # Optional. The candidate to inference config map for the evaluation run. The candidate can be up to 128 characters long and can consist of any UTF-8 characters.
+ "a_key": { # An inference config used for model inference during the evaluation run.
+ "generationConfig": { # Generation config. # Optional. Generation config.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "model": "A String", # Required. The fully qualified name of the publisher model or endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ },
+ },
+ "labels": { # Optional. Labels for the evaluation run.
+ "a_key": "A String",
+ },
+ "metadata": "", # Optional. Metadata about the evaluation run, can be used by the caller to store additional tracking information about the evaluation run.
+ "name": "A String", # Identifier. The resource name of the EvaluationRun. This is a unique identifier. Format: `projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}`
+ "state": "A String", # Output only. The state of the evaluation run.
+}
+list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
+ Lists Evaluation Runs.
+
+Args:
+ parent: string, Required. The resource name of the Location from which to list the Evaluation Runs. Format: `projects/{project}/locations/{location}` (required)
+ filter: string, Optional. Filter expression that matches a subset of the EvaluationRuns to show. For field names both snake_case and camelCase are supported. For more information about filter syntax, see [AIP-160](https://google.aip.dev/160).
+ orderBy: string, Optional. A comma-separated list of fields to order by, sorted in ascending order by default. Use `desc` after a field name for descending.
+ pageSize: integer, Optional. The maximum number of Evaluation Runs to return.
+ pageToken: string, Optional. A page token, received from a previous `ListEvaluationRuns` call. Provide this to retrieve the subsequent page.
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # Response message for EvaluationManagementService.ListEvaluationRuns.
+ "evaluationRuns": [ # List of EvaluationRuns in the requested page.
+ { # EvaluationRun is a resource that represents a single evaluation run, which includes a set of prompts, model responses, evaluation configuration and the resulting metrics.
+ "completionTime": "A String", # Output only. Time when the evaluation run was completed.
+ "createTime": "A String", # Output only. Time when the evaluation run was created.
+ "dataSource": { # The data source for the evaluation run. # Required. The data source for the evaluation run.
+ "bigqueryRequestSet": { # The request set for the evaluation run. # Evaluation data in bigquery.
+ "candidateResponseColumns": { # Optional. Map of candidate name to candidate response column name. The column will be in evaluation_item.CandidateResponse format.
+ "a_key": "A String",
+ },
+ "promptColumn": "A String", # Optional. The name of the column that contains the requests to evaluate. This will be in evaluation_item.EvalPrompt format.
+ "rubricsColumn": "A String", # Optional. The name of the column that contains the rubrics. This will be in evaluation_rubric.RubricGroup format (cl/762595858).
+ "samplingConfig": { # The sampling config. # Optional. The sampling config for the bigquery resource.
+ "samplingCount": 42, # Optional. The total number of logged data to import. If available data is less than the sampling count, all data will be imported. Default is 100.
+ "samplingDuration": "A String", # Optional. How long to wait before sampling data from the BigQuery table. If not specified, defaults to 0.
+ "samplingMethod": "A String", # Optional. The sampling method to use.
+ },
+ "uri": "A String", # Required. The URI of a BigQuery table. e.g. bq://projectId.bqDatasetId.bqTableId
+ },
+ "evaluationSet": "A String", # The EvaluationSet resource name. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}`
+ },
+ "displayName": "A String", # Required. The display name of the Evaluation Run.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Output only. Only populated when the evaluation run's state is FAILED or CANCELLED.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "evaluationConfig": { # The Evalution configuration used for the evaluation run. # Required. The configuration used for the evaluation.
+ "autoraterConfig": { # The autorater config used for the evaluation run. # Optional. The autorater config for the evaluation run.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metrics": [ # Required. The metrics to be calculated in the evaluation run.
+ { # The metric used for evaluation runs.
+ "computationBasedMetricSpec": { # Specification for a computation based metric. # Spec for a computation based metric.
+ "parameters": { # Optional. A map of parameters for the metric, e.g. {"rouge_type": "rougeL"}.
+ "a_key": "", # Properties of the object.
+ },
+ "type": "A String", # Required. The type of the computation based metric.
+ },
+ "llmBasedMetricSpec": { # Specification for an LLM based metric. # Spec for an LLM based metric.
+ "additionalConfig": { # Optional. Optional additional configuration for the metric.
+ "a_key": "", # Properties of the object.
+ },
+ "judgeAutoraterConfig": { # The autorater config used for the evaluation run. # Optional. Optional configuration for the judge LLM (Autorater).
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metricPromptTemplate": "A String", # Required. Template for the prompt sent to the judge model.
+ "predefinedRubricGenerationSpec": { # Specification for a pre-defined metric. # Dynamically generate rubrics using a predefined spec.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input. Refers to a key in the rubric_groups map of EvaluationInstance.
+ "systemInstruction": "A String", # Optional. System instructions for the judge model.
+ },
+ "metric": "A String", # Required. The name of the metric.
+ "predefinedMetricSpec": { # Specification for a pre-defined metric. # Spec for a pre-defined metric.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricBasedMetricSpec": { # Specification for a metric that is based on rubrics. # Spec for rubric based metric.
+ "inlineRubrics": { # Defines a list of rubrics, used when providing rubrics inline. # Use rubrics provided directly in the spec.
+ "rubrics": [ # The list of rubrics.
+ { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics.
+ "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric.
+ "property": { # Defines criteria based on a specific property. # Evaluation criteria based on a specific property.
+ "description": "A String", # Description of the property being evaluated. Example: "The model's response is grammatically correct."
+ },
+ },
+ "importance": "A String", # Optional. The relative importance of this rubric.
+ "rubricId": "A String", # Unique identifier for the rubric. This ID is used to refer to this rubric, e.g., in RubricVerdict.
+ "type": "A String", # Optional. A type designator for the rubric, which can inform how it's evaluated or interpreted by systems or users. It's recommended to use consistent, well-defined, upper snake_case strings. Examples: "SUMMARIZATION_QUALITY", "SAFETY_HARMFUL_CONTENT", "INSTRUCTION_ADHERENCE".
+ },
+ ],
+ },
+ "judgeAutoraterConfig": { # The autorater config used for the evaluation run. # Optional. Optional configuration for the judge LLM (Autorater). The definition of AutoraterConfig needs to be provided.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "metricPromptTemplate": "A String", # Optional. Template for the prompt used by the judge model to evaluate against rubrics.
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics for evaluation using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Use a pre-defined group of rubrics associated with the input content. This refers to a key in the `rubric_groups` map of `RubricEnhancedContents`.
+ },
+ },
+ ],
+ "outputConfig": { # The output config for the evaluation run. # Optional. The output config for the evaluation run.
+ "bigqueryDestination": { # The BigQuery location for the output content. # BigQuery destination for evaluation output.
+ "outputUri": "A String", # Required. BigQuery URI to a project or table, up to 2000 characters long. When only the project is specified, the Dataset and Table is created. When the full table reference is specified, the Dataset must exist and table must not exist. Accepted forms: * BigQuery path. For example: `bq://projectId` or `bq://projectId.bqDatasetId` or `bq://projectId.bqDatasetId.bqTableId`.
+ },
+ "gcsDestination": { # The Google Cloud Storage location where the output is to be written to. # Cloud Storage destination for evaluation output.
+ "outputUriPrefix": "A String", # Required. Google Cloud Storage URI to output directory. If the uri doesn't end with '/', a '/' will be automatically appended. The directory is created if it doesn't exist.
+ },
+ },
+ "promptTemplate": { # Prompt template used for inference. # The prompt template used for inference. The values for variables in the prompt template are defined in EvaluationItem.EvaluationPrompt.PromptTemplateData.values.
+ "gcsUri": "A String", # Prompt template stored in Cloud Storage. Format: "gs://my-bucket/file-name.txt".
+ "promptTemplate": "A String", # Inline prompt template. Template variables should be in the format "{var_name}". Example: "Translate the following from {source_lang} to {target_lang}: {text}"
+ },
+ "rubricConfigs": [ # Optional. The rubric configs for the evaluation run. They are used to generate rubrics which can be used by rubric-based metrics. Multiple rubric configs can be specified for rubric generation but only one rubric config can be used for a rubric-based metric. If more than one rubric config is provided, the evaluation metric must specify a rubric group key. Note that if a generation spec is specified on both a rubric config and an evaluation metric, the rubrics generated for the metric will be used for evaluation.
+ { # Configuration for a rubric group to be generated/saved for evaluation.
+ "predefinedRubricGenerationSpec": { # Specification for a pre-defined metric. # Dynamically generate rubrics using a predefined spec.
+ "metricSpecName": "A String", # Required. The name of a pre-defined metric, such as "instruction_following_v1" or "text_quality_v1".
+ "parameters": { # Optional. The parameters needed to run the pre-defined metric.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "rubricGenerationSpec": { # Specification for how rubrics should be generated. # Dynamically generate rubrics using this specification.
+ "modelConfig": { # The autorater config used for the evaluation run. # Optional. Configuration for the model used in rubric generation. Configs including sampling count and base model can be specified here. Flipping is not supported for rubric generation.
+ "autoraterModel": "A String", # Optional. The fully qualified name of the publisher model or tuned autorater endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ "generationConfig": { # Generation config. # Optional. Configuration options for model generation and outputs.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "sampleCount": 42, # Optional. Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.
+ },
+ "promptTemplate": "A String", # Optional. Template for the prompt used to generate rubrics. The details should be updated based on the most-recent recipe requirements.
+ "rubricContentType": "A String", # Optional. The type of rubric content to be generated.
+ "rubricTypeOntology": [ # Optional. An optional, pre-defined list of allowed types for generated rubrics. If this field is provided, it implies `include_rubric_type` should be true, and the generated rubric types should be chosen from this ontology.
+ "A String",
+ ],
+ },
+ "rubricGroupKey": "A String", # Required. The key used to save the generated rubrics. If a generation spec is provided, this key will be used for the name of the generated rubric group. Otherwise, this key will be used to look up the existing rubric group on the evaluation item. Note that if a rubric group key is specified on both a rubric config and an evaluation metric, the key from the metric will be used to select the rubrics for evaluation.
+ },
+ ],
+ },
+ "evaluationResults": { # The results of the evaluation run. # Output only. The results of the evaluation run. Only populated when the evaluation run's state is SUCCEEDED.
+ "evaluationSet": "A String", # The evaluation set where item level results are stored.
+ "summaryMetrics": { # The summary metrics for the evaluation run. # Optional. The summary metrics for the evaluation run.
+ "failedItems": 42, # Optional. The number of items that failed to be evaluated.
+ "metrics": { # Optional. Map of metric name to metric value.
+ "a_key": "",
+ },
+ "totalItems": 42, # Optional. The total number of items that were evaluated.
+ },
+ },
+ "evaluationSetSnapshot": "A String", # Output only. The specific evaluation set of the evaluation run. For runs with an evaluation set input, this will be that same set. For runs with BigQuery input, it's the sampled BigQuery dataset.
+ "inferenceConfigs": { # Optional. The candidate to inference config map for the evaluation run. The candidate can be up to 128 characters long and can consist of any UTF-8 characters.
+ "a_key": { # An inference config used for model inference during the evaluation run.
+ "generationConfig": { # Generation config. # Optional. Generation config.
+ "audioTimestamp": True or False, # Optional. If enabled, audio timestamp will be included in the request to the model.
+ "candidateCount": 42, # Optional. Number of candidates to generate.
+ "enableAffectiveDialog": True or False, # Optional. If enabled, the model will detect emotions and adapt its responses accordingly.
+ "frequencyPenalty": 3.14, # Optional. Frequency penalties.
+ "logprobs": 42, # Optional. Logit probabilities.
+ "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
+ "mediaResolution": "A String", # Optional. If specified, the media resolution specified will be used.
+ "presencePenalty": 3.14, # Optional. Positive penalties.
+ "responseJsonSchema": "", # Optional. Output schema of the generated response. This is an alternative to `response_schema` that accepts [JSON Schema](https://json-schema.org/). If set, `response_schema` must be omitted, but `response_mime_type` is required. While the full JSON Schema may be sent, not all features are supported. Specifically, only the following properties are supported: - `$id` - `$defs` - `$ref` - `$anchor` - `type` - `format` - `title` - `description` - `enum` (for strings and numbers) - `items` - `prefixItems` - `minItems` - `maxItems` - `minimum` - `maximum` - `anyOf` - `oneOf` (interpreted the same as `anyOf`) - `properties` - `additionalProperties` - `required` The non-standard `propertyOrdering` property may also be set. Cyclic references are unrolled to a limited degree and, as such, may only be used within non-required properties. (Nullable properties are not sufficient.) If `$ref` is set on a sub-schema, no other properties, except for than those starting as a `$`, may be set.
+ "responseLogprobs": True or False, # Optional. If true, export the logprobs results in response.
+ "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
+ "responseModalities": [ # Optional. The modalities of the response.
+ "A String",
+ ],
+ "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
+ "additionalProperties": "", # Optional. Can either be a boolean or an object; controls the presence of additional properties.
+ "anyOf": [ # Optional. The value should be validated against any (one or more) of the subschemas in the list.
+ # Object with schema name: GoogleCloudAiplatformV1Schema
+ ],
+ "default": "", # Optional. Default value of the data.
+ "defs": { # Optional. A map of definitions for use by `ref` Only allowed at the root of the schema.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "description": "A String", # Optional. The description of the data.
+ "enum": [ # Optional. Possible values of the element of primitive type with enum format. Examples: 1. We can define direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]} 2. We can define apartment number as : {type:INTEGER, format:enum, enum:["101", "201", "301"]}
+ "A String",
+ ],
+ "example": "", # Optional. Example of the object. Will only populated when the object is the root.
+ "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
+ "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
+ "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
+ "maxLength": "A String", # Optional. Maximum length of the Type.STRING
+ "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
+ "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
+ "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
+ "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
+ "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
+ "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
+ "nullable": True or False, # Optional. Indicates if the value may be null.
+ "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
+ "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
+ "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
+ },
+ "propertyOrdering": [ # Optional. The order of the properties. Not a standard field in open api spec. Only used to support the order of the properties.
+ "A String",
+ ],
+ "ref": "A String", # Optional. Allows indirect references between schema nodes. The value should be a valid reference to a child of the root `defs`. For example, the following schema defines a reference to a schema node named "Pet": type: object properties: pet: ref: #/defs/Pet defs: Pet: type: object properties: name: type: string The value of the "pet" property is a reference to the schema node named "Pet". See details in https://json-schema.org/understanding-json-schema/structuring
+ "required": [ # Optional. Required properties of Type.OBJECT.
+ "A String",
+ ],
+ "title": "A String", # Optional. The title of the Schema.
+ "type": "A String", # Optional. The type of the data.
+ },
+ "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
+ "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
+ "modelRoutingPreference": "A String", # The model routing preference.
+ },
+ "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
+ "modelName": "A String", # The model name to use. Only the public LLM models are accepted. See [Supported models](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported-models).
+ },
+ },
+ "seed": 42, # Optional. Seed.
+ "speechConfig": { # The speech generation config. # Optional. The speech generation config.
+ "languageCode": "A String", # Optional. Language code (ISO 639. e.g. en-US) for the speech synthesization.
+ "voiceConfig": { # The configuration for the voice to use. # The configuration for the speaker to use.
+ "prebuiltVoiceConfig": { # The configuration for the prebuilt speaker to use. # The configuration for the prebuilt voice to use.
+ "voiceName": "A String", # The name of the preset voice to use.
+ },
+ },
+ },
+ "stopSequences": [ # Optional. Stop sequences.
+ "A String",
+ ],
+ "temperature": 3.14, # Optional. Controls the randomness of predictions.
+ "thinkingConfig": { # Config for thinking features. # Optional. Config for thinking features. An error will be returned if this field is set for models that don't support thinking.
+ "includeThoughts": True or False, # Optional. Indicates whether to include thoughts in the response. If true, thoughts are returned only when available.
+ "thinkingBudget": 42, # Optional. Indicates the thinking budget in tokens.
+ },
+ "topK": 3.14, # Optional. If specified, top-k sampling will be used.
+ "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
+ },
+ "model": "A String", # Required. The fully qualified name of the publisher model or endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}`
+ },
+ },
+ "labels": { # Optional. Labels for the evaluation run.
+ "a_key": "A String",
+ },
+ "metadata": "", # Optional. Metadata about the evaluation run, can be used by the caller to store additional tracking information about the evaluation run.
+ "name": "A String", # Identifier. The resource name of the EvaluationRun. This is a unique identifier. Format: `projects/{project}/locations/{location}/evaluationRuns/{evaluation_run}`
+ "state": "A String", # Output only. The state of the evaluation run.
+ },
+ ],
+ "nextPageToken": "A String", # A token to retrieve the next page of results.
+}
+list_next()
+ Retrieves the next page of results. + + Args: + previous_request: The request for the previous page. (required) + previous_response: The response from the request for the previous page. (required) + + Returns: + A request object that you can call 'execute()' on to request the next + page. Returns None if there are no more items in the collection. ++
+ close()
Close httplib2 connections.
+
+ create(parent, body=None, x__xgafv=None)
Creates an Evaluation Set.
+ +Deletes an Evaluation Set.
+ +Gets an Evaluation Set.
+
+ list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
Lists Evaluation Sets.
+ +Retrieves the next page of results.
+
+ patch(name, body=None, updateMask=None, x__xgafv=None)
Updates an Evaluation Set.
+close()
+ Close httplib2 connections.+
create(parent, body=None, x__xgafv=None)
+ Creates an Evaluation Set.
+
+Args:
+ parent: string, Required. The resource name of the Location to create the Evaluation Set in. Format: `projects/{project}/locations/{location}` (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # EvaluationSet is a collection of related EvaluationItems that are evaluated together.
+ "createTime": "A String", # Output only. Timestamp when this item was created.
+ "displayName": "A String", # Required. The display name of the EvaluationSet.
+ "evaluationItems": [ # Required. The EvaluationItems that are part of this dataset.
+ "A String",
+ ],
+ "metadata": "", # Optional. Metadata for the EvaluationSet.
+ "name": "A String", # Identifier. The resource name of the EvaluationSet. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}`
+ "updateTime": "A String", # Output only. Timestamp when this item was last updated.
+}
+
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # EvaluationSet is a collection of related EvaluationItems that are evaluated together.
+ "createTime": "A String", # Output only. Timestamp when this item was created.
+ "displayName": "A String", # Required. The display name of the EvaluationSet.
+ "evaluationItems": [ # Required. The EvaluationItems that are part of this dataset.
+ "A String",
+ ],
+ "metadata": "", # Optional. Metadata for the EvaluationSet.
+ "name": "A String", # Identifier. The resource name of the EvaluationSet. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}`
+ "updateTime": "A String", # Output only. Timestamp when this item was last updated.
+}
+delete(name, x__xgafv=None)
+ Deletes an Evaluation Set.
+
+Args:
+ name: string, Required. The name of the EvaluationSet resource to be deleted. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}` (required)
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # This resource represents a long-running operation that is the result of a network API call.
+ "done": True or False, # If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available.
+ "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation.
+ "code": 42, # The status code, which should be an enum value of google.rpc.Code.
+ "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
+ {
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ ],
+ "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
+ },
+ "metadata": { # Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any.
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+ "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the `name` should be a resource name ending with `operations/{unique_id}`.
+ "response": { # The normal, successful response of the operation. If the original method returns no data on success, such as `Delete`, the response is `google.protobuf.Empty`. If the original method is standard `Get`/`Create`/`Update`, the response should be the resource. For other methods, the response should have the type `XxxResponse`, where `Xxx` is the original method name. For example, if the original method name is `TakeSnapshot()`, the inferred response type is `TakeSnapshotResponse`.
+ "a_key": "", # Properties of the object. Contains field @type with type URL.
+ },
+}
+get(name, x__xgafv=None)
+ Gets an Evaluation Set.
+
+Args:
+ name: string, Required. The name of the EvaluationSet resource. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}` (required)
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # EvaluationSet is a collection of related EvaluationItems that are evaluated together.
+ "createTime": "A String", # Output only. Timestamp when this item was created.
+ "displayName": "A String", # Required. The display name of the EvaluationSet.
+ "evaluationItems": [ # Required. The EvaluationItems that are part of this dataset.
+ "A String",
+ ],
+ "metadata": "", # Optional. Metadata for the EvaluationSet.
+ "name": "A String", # Identifier. The resource name of the EvaluationSet. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}`
+ "updateTime": "A String", # Output only. Timestamp when this item was last updated.
+}
+list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
+ Lists Evaluation Sets.
+
+Args:
+ parent: string, Required. The resource name of the Location from which to list the Evaluation Sets. Format: `projects/{project}/locations/{location}` (required)
+ filter: string, Optional. Filter expression that matches a subset of the EvaluationSets to show. For field names both snake_case and camelCase are supported. For more information about filter syntax, see [AIP-160](https://google.aip.dev/160).
+ orderBy: string, Optional. A comma-separated list of fields to order by, sorted in ascending order by default. Use `desc` after a field name for descending.
+ pageSize: integer, Optional. The maximum number of Evaluation Sets to return.
+ pageToken: string, Optional. A page token, received from a previous `ListEvaluationSets` call. Provide this to retrieve the subsequent page.
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # Response message for EvaluationManagementService.ListEvaluationSets.
+ "evaluationSets": [ # List of EvaluationSets in the requested page.
+ { # EvaluationSet is a collection of related EvaluationItems that are evaluated together.
+ "createTime": "A String", # Output only. Timestamp when this item was created.
+ "displayName": "A String", # Required. The display name of the EvaluationSet.
+ "evaluationItems": [ # Required. The EvaluationItems that are part of this dataset.
+ "A String",
+ ],
+ "metadata": "", # Optional. Metadata for the EvaluationSet.
+ "name": "A String", # Identifier. The resource name of the EvaluationSet. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}`
+ "updateTime": "A String", # Output only. Timestamp when this item was last updated.
+ },
+ ],
+ "nextPageToken": "A String", # A token to retrieve the next page of results.
+}
+list_next()
+ Retrieves the next page of results. + + Args: + previous_request: The request for the previous page. (required) + previous_response: The response from the request for the previous page. (required) + + Returns: + A request object that you can call 'execute()' on to request the next + page. Returns None if there are no more items in the collection. ++
patch(name, body=None, updateMask=None, x__xgafv=None)
+ Updates an Evaluation Set.
+
+Args:
+ name: string, Identifier. The resource name of the EvaluationSet. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}` (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # EvaluationSet is a collection of related EvaluationItems that are evaluated together.
+ "createTime": "A String", # Output only. Timestamp when this item was created.
+ "displayName": "A String", # Required. The display name of the EvaluationSet.
+ "evaluationItems": [ # Required. The EvaluationItems that are part of this dataset.
+ "A String",
+ ],
+ "metadata": "", # Optional. Metadata for the EvaluationSet.
+ "name": "A String", # Identifier. The resource name of the EvaluationSet. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}`
+ "updateTime": "A String", # Output only. Timestamp when this item was last updated.
+}
+
+ updateMask: string, Optional. The update mask applies to the resource. For the `FieldMask` definition, see google.protobuf.FieldMask.
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # EvaluationSet is a collection of related EvaluationItems that are evaluated together.
+ "createTime": "A String", # Output only. Timestamp when this item was created.
+ "displayName": "A String", # Required. The display name of the EvaluationSet.
+ "evaluationItems": [ # Required. The EvaluationItems that are part of this dataset.
+ "A String",
+ ],
+ "metadata": "", # Optional. Metadata for the EvaluationSet.
+ "name": "A String", # Identifier. The resource name of the EvaluationSet. Format: `projects/{project}/locations/{location}/evaluationSets/{evaluation_set}`
+ "updateTime": "A String", # Output only. Timestamp when this item was last updated.
+}
+Returns the endpoints Resource.
+ +Returns the evaluationItems Resource.
+ +
+ evaluationRuns()
+
Returns the evaluationRuns Resource.
+ +
+ evaluationSets()
+
Returns the evaluationSets Resource.
+ @@ -265,6 +280,9 @@
generateInstanceRubrics(location, body=None, x__xgafv=None)
Generates rubrics for a given prompt. A rubric represents a single testable criterion for evaluation. One input prompt could have multiple rubrics This RPC allows users to get suggested rubrics based on provided prompt, which can then be reviewed and used for subsequent evaluations.
+
+ generateSyntheticData(location, body=None, x__xgafv=None)
Generates synthetic data based on the provided configuration.
Gets information about a location.
@@ -1732,8 +1750,17 @@generateSyntheticData(location, body=None, x__xgafv=None)
+ Generates synthetic data based on the provided configuration.
+
+Args:
+ location: string, Required. The resource name of the Location to run the job. Format: `projects/{project}/locations/{location}` (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # Request message for DataFoundryService.GenerateSyntheticData.
+ "count": 42, # Required. The number of synthetic examples to generate. For this stateless API, the count is limited to a small number.
+ "examples": [ # Optional. A list of few-shot examples to guide the model's output style and format.
+ { # Represents a single synthetic example, composed of multiple fields. Used for providing few-shot examples in the request and for returning generated examples in the response.
+ "fields": [ # Required. A list of fields that constitute an example.
+ { # Represents a single named field within a SyntheticExample.
+ "content": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn. # Required. The content of the field.
+ "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
+ { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
+ "codeExecutionResult": { # Result of executing the [ExecutableCode]. Only generated when using the [CodeExecution] tool, and always follows a `part` containing the [ExecutableCode]. # Optional. Result of executing the [ExecutableCode].
+ "outcome": "A String", # Required. Outcome of the code execution.
+ "output": "A String", # Optional. Contains stdout when code execution is successful, stderr or other description otherwise.
+ },
+ "executableCode": { # Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated. # Optional. Code generated by the model that is meant to be executed.
+ "code": "A String", # Required. The code to be executed.
+ "language": "A String", # Required. Programming language of the `code`.
+ },
+ "fileData": { # URI based data. # Optional. URI based data.
+ "displayName": "A String", # Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "fileUri": "A String", # Required. URI.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
+ "args": { # Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
+ "a_key": "", # Properties of the object.
+ },
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
+ },
+ "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
+ "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "inlineData": { # Content blob. # Optional. Inlined bytes data.
+ "data": "A String", # Required. Raw bytes.
+ "displayName": "A String", # Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "text": "A String", # Optional. Text part (can be code).
+ "thought": True or False, # Optional. Indicates if the part is thought from the model.
+ "thoughtSignature": "A String", # Optional. An opaque signature for the thought so it can be reused in subsequent requests.
+ "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
+ "endOffset": "A String", # Optional. The end offset of the video.
+ "fps": 3.14, # Optional. The frame rate of the video sent to the model. If not specified, the default value will be 1.0. The fps range is (0.0, 24.0].
+ "startOffset": "A String", # Optional. The start offset of the video.
+ },
+ },
+ ],
+ "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
+ },
+ "fieldName": "A String", # Optional. The name of the field.
+ },
+ ],
+ },
+ ],
+ "outputFieldSpecs": [ # Required. The schema of the desired output, defined by a list of fields.
+ { # Defines a specification for a single output field.
+ "fieldName": "A String", # Required. The name of the output field.
+ "fieldType": "A String", # Optional. The data type of the field. Defaults to CONTENT if not set.
+ "guidance": "A String", # Optional. Optional, but recommended. Additional guidance specific to this field to provide targeted instructions for the LLM to generate the content of a single output field. While the LLM can sometimes infer content from the field name, providing explicit guidance is preferred.
+ },
+ ],
+ "taskDescription": { # Defines a generation strategy based on a high-level task description. # Generate data from a high-level task description.
+ "taskDescription": "A String", # Required. A high-level description of the synthetic data to be generated.
+ },
+}
+
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # The response containing the generated data.
+ "syntheticExamples": [ # A list of generated synthetic examples.
+ { # Represents a single synthetic example, composed of multiple fields. Used for providing few-shot examples in the request and for returning generated examples in the response.
+ "fields": [ # Required. A list of fields that constitute an example.
+ { # Represents a single named field within a SyntheticExample.
+ "content": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn. # Required. The content of the field.
+ "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
+ { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
+ "codeExecutionResult": { # Result of executing the [ExecutableCode]. Only generated when using the [CodeExecution] tool, and always follows a `part` containing the [ExecutableCode]. # Optional. Result of executing the [ExecutableCode].
+ "outcome": "A String", # Required. Outcome of the code execution.
+ "output": "A String", # Optional. Contains stdout when code execution is successful, stderr or other description otherwise.
+ },
+ "executableCode": { # Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated. # Optional. Code generated by the model that is meant to be executed.
+ "code": "A String", # Required. The code to be executed.
+ "language": "A String", # Required. Programming language of the `code`.
+ },
+ "fileData": { # URI based data. # Optional. URI based data.
+ "displayName": "A String", # Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "fileUri": "A String", # Required. URI.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
+ "args": { # Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
+ "a_key": "", # Properties of the object.
+ },
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
+ },
+ "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
+ "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
+ "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
+ "a_key": "", # Properties of the object.
+ },
+ },
+ "inlineData": { # Content blob. # Optional. Inlined bytes data.
+ "data": "A String", # Required. Raw bytes.
+ "displayName": "A String", # Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_execution, google_search, and url_context) are enabled.
+ "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
+ },
+ "text": "A String", # Optional. Text part (can be code).
+ "thought": True or False, # Optional. Indicates if the part is thought from the model.
+ "thoughtSignature": "A String", # Optional. An opaque signature for the thought so it can be reused in subsequent requests.
+ "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
+ "endOffset": "A String", # Optional. The end offset of the video.
+ "fps": 3.14, # Optional. The frame rate of the video sent to the model. If not specified, the default value will be 1.0. The fps range is (0.0, 24.0].
+ "startOffset": "A String", # Optional. The start offset of the video.
+ },
+ },
+ ],
+ "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
+ },
+ "fieldName": "A String", # Optional. The name of the field.
+ },
+ ],
+ },
+ ],
+}
+get(name, x__xgafv=None)
Gets information about a location. diff --git a/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.html b/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.html index ad70427756b..47efd6971ad 100644 --- a/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.html +++ b/docs/dyn/aiplatform_v1.projects.locations.reasoningEngines.html @@ -129,6 +129,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". @@ -268,6 +271,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". @@ -349,6 +355,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". @@ -435,6 +444,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". diff --git a/docs/dyn/aiplatform_v1.reasoningEngines.html b/docs/dyn/aiplatform_v1.reasoningEngines.html index 7116864cab2..389a4ccff45 100644 --- a/docs/dyn/aiplatform_v1.reasoningEngines.html +++ b/docs/dyn/aiplatform_v1.reasoningEngines.html @@ -123,6 +123,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". @@ -263,6 +266,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". @@ -344,6 +350,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". @@ -430,6 +439,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.deploymentResourcePools.html b/docs/dyn/aiplatform_v1beta1.projects.locations.deploymentResourcePools.html index bc3d7e295a0..65341810955 100644 --- a/docs/dyn/aiplatform_v1beta1.projects.locations.deploymentResourcePools.html +++ b/docs/dyn/aiplatform_v1beta1.projects.locations.deploymentResourcePools.html @@ -523,6 +523,7 @@Method Details
"requiredReplicaCount": 42, # Optional. Number of required available replicas for the deployment to succeed. This field is only needed when partial deployment/mutation is desired. If set, the deploy/mutate operation will succeed once available_replica_count reaches required_replica_count, and the rest of the replicas will be retried. If not set, the default required_replica_count will be min_replica_count. "spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms). }, + "disableContainerLogging": True or False, # For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true. "disableExplanations": True or False, # If true, deploy the model without explainable feature, regardless the existence of Model.explanation_spec or explanation_spec. "displayName": "A String", # The display name of the DeployedModel. If not provided upon creation, the Model's display_name is used. "enableAccessLogging": True or False, # If true, online prediction access logs are sent to Cloud Logging. These logs are like standard server access logs, containing information like timestamp and latency for each prediction request. Note that logs may incur a cost, especially if your project receives prediction requests at a high queries per second rate (QPS). Estimate your costs before enabling this option. diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.endpoints.html b/docs/dyn/aiplatform_v1beta1.projects.locations.endpoints.html index c27806b555b..30b81327ec3 100644 --- a/docs/dyn/aiplatform_v1beta1.projects.locations.endpoints.html +++ b/docs/dyn/aiplatform_v1beta1.projects.locations.endpoints.html @@ -735,6 +735,7 @@Method Details
"requiredReplicaCount": 42, # Optional. Number of required available replicas for the deployment to succeed. This field is only needed when partial deployment/mutation is desired. If set, the deploy/mutate operation will succeed once available_replica_count reaches required_replica_count, and the rest of the replicas will be retried. If not set, the default required_replica_count will be min_replica_count. "spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms). }, + "disableContainerLogging": True or False, # For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true. "disableExplanations": True or False, # If true, deploy the model without explainable feature, regardless the existence of Model.explanation_spec or explanation_spec. "displayName": "A String", # The display name of the DeployedModel. If not provided upon creation, the Model's display_name is used. "enableAccessLogging": True or False, # If true, online prediction access logs are sent to Cloud Logging. These logs are like standard server access logs, containing information like timestamp and latency for each prediction request. Note that logs may incur a cost, especially if your project receives prediction requests at a high queries per second rate (QPS). Estimate your costs before enabling this option. @@ -1065,6 +1066,7 @@Method Details
"requiredReplicaCount": 42, # Optional. Number of required available replicas for the deployment to succeed. This field is only needed when partial deployment/mutation is desired. If set, the deploy/mutate operation will succeed once available_replica_count reaches required_replica_count, and the rest of the replicas will be retried. If not set, the default required_replica_count will be min_replica_count. "spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms). }, + "disableContainerLogging": True or False, # For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true. "disableExplanations": True or False, # If true, deploy the model without explainable feature, regardless the existence of Model.explanation_spec or explanation_spec. "displayName": "A String", # The display name of the DeployedModel. If not provided upon creation, the Model's display_name is used. "enableAccessLogging": True or False, # If true, online prediction access logs are sent to Cloud Logging. These logs are like standard server access logs, containing information like timestamp and latency for each prediction request. Note that logs may incur a cost, especially if your project receives prediction requests at a high queries per second rate (QPS). Estimate your costs before enabling this option. @@ -2523,6 +2525,7 @@Method Details
"requiredReplicaCount": 42, # Optional. Number of required available replicas for the deployment to succeed. This field is only needed when partial deployment/mutation is desired. If set, the deploy/mutate operation will succeed once available_replica_count reaches required_replica_count, and the rest of the replicas will be retried. If not set, the default required_replica_count will be min_replica_count. "spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms). }, + "disableContainerLogging": True or False, # For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true. "disableExplanations": True or False, # If true, deploy the model without explainable feature, regardless the existence of Model.explanation_spec or explanation_spec. "displayName": "A String", # The display name of the DeployedModel. If not provided upon creation, the Model's display_name is used. "enableAccessLogging": True or False, # If true, online prediction access logs are sent to Cloud Logging. These logs are like standard server access logs, containing information like timestamp and latency for each prediction request. Note that logs may incur a cost, especially if your project receives prediction requests at a high queries per second rate (QPS). Estimate your costs before enabling this option. @@ -2843,6 +2846,7 @@Method Details
"requiredReplicaCount": 42, # Optional. Number of required available replicas for the deployment to succeed. This field is only needed when partial deployment/mutation is desired. If set, the deploy/mutate operation will succeed once available_replica_count reaches required_replica_count, and the rest of the replicas will be retried. If not set, the default required_replica_count will be min_replica_count. "spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms). }, + "disableContainerLogging": True or False, # For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true. "disableExplanations": True or False, # If true, deploy the model without explainable feature, regardless the existence of Model.explanation_spec or explanation_spec. "displayName": "A String", # The display name of the DeployedModel. If not provided upon creation, the Model's display_name is used. "enableAccessLogging": True or False, # If true, online prediction access logs are sent to Cloud Logging. These logs are like standard server access logs, containing information like timestamp and latency for each prediction request. Note that logs may incur a cost, especially if your project receives prediction requests at a high queries per second rate (QPS). Estimate your costs before enabling this option. @@ -3126,6 +3130,7 @@Method Details
"requiredReplicaCount": 42, # Optional. Number of required available replicas for the deployment to succeed. This field is only needed when partial deployment/mutation is desired. If set, the deploy/mutate operation will succeed once available_replica_count reaches required_replica_count, and the rest of the replicas will be retried. If not set, the default required_replica_count will be min_replica_count. "spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms). }, + "disableContainerLogging": True or False, # For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true. "disableExplanations": True or False, # If true, deploy the model without explainable feature, regardless the existence of Model.explanation_spec or explanation_spec. "displayName": "A String", # The display name of the DeployedModel. If not provided upon creation, the Model's display_name is used. "enableAccessLogging": True or False, # If true, online prediction access logs are sent to Cloud Logging. These logs are like standard server access logs, containing information like timestamp and latency for each prediction request. Note that logs may incur a cost, especially if your project receives prediction requests at a high queries per second rate (QPS). Estimate your costs before enabling this option. @@ -3373,6 +3378,7 @@Method Details
"requiredReplicaCount": 42, # Optional. Number of required available replicas for the deployment to succeed. This field is only needed when partial deployment/mutation is desired. If set, the deploy/mutate operation will succeed once available_replica_count reaches required_replica_count, and the rest of the replicas will be retried. If not set, the default required_replica_count will be min_replica_count. "spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms). }, + "disableContainerLogging": True or False, # For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true. "disableExplanations": True or False, # If true, deploy the model without explainable feature, regardless the existence of Model.explanation_spec or explanation_spec. "displayName": "A String", # The display name of the DeployedModel. If not provided upon creation, the Model's display_name is used. "enableAccessLogging": True or False, # If true, online prediction access logs are sent to Cloud Logging. These logs are like standard server access logs, containing information like timestamp and latency for each prediction request. Note that logs may incur a cost, especially if your project receives prediction requests at a high queries per second rate (QPS). Estimate your costs before enabling this option. @@ -3645,6 +3651,7 @@Method Details
"requiredReplicaCount": 42, # Optional. Number of required available replicas for the deployment to succeed. This field is only needed when partial deployment/mutation is desired. If set, the deploy/mutate operation will succeed once available_replica_count reaches required_replica_count, and the rest of the replicas will be retried. If not set, the default required_replica_count will be min_replica_count. "spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms). }, + "disableContainerLogging": True or False, # For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true. "disableExplanations": True or False, # If true, deploy the model without explainable feature, regardless the existence of Model.explanation_spec or explanation_spec. "displayName": "A String", # The display name of the DeployedModel. If not provided upon creation, the Model's display_name is used. "enableAccessLogging": True or False, # If true, online prediction access logs are sent to Cloud Logging. These logs are like standard server access logs, containing information like timestamp and latency for each prediction request. Note that logs may incur a cost, especially if your project receives prediction requests at a high queries per second rate (QPS). Estimate your costs before enabling this option. @@ -5067,6 +5074,7 @@Method Details
"requiredReplicaCount": 42, # Optional. Number of required available replicas for the deployment to succeed. This field is only needed when partial deployment/mutation is desired. If set, the deploy/mutate operation will succeed once available_replica_count reaches required_replica_count, and the rest of the replicas will be retried. If not set, the default required_replica_count will be min_replica_count. "spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms). }, + "disableContainerLogging": True or False, # For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true. "disableExplanations": True or False, # If true, deploy the model without explainable feature, regardless the existence of Model.explanation_spec or explanation_spec. "displayName": "A String", # The display name of the DeployedModel. If not provided upon creation, the Model's display_name is used. "enableAccessLogging": True or False, # If true, online prediction access logs are sent to Cloud Logging. These logs are like standard server access logs, containing information like timestamp and latency for each prediction request. Note that logs may incur a cost, especially if your project receives prediction requests at a high queries per second rate (QPS). Estimate your costs before enabling this option. diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.html b/docs/dyn/aiplatform_v1beta1.projects.locations.html index a45ba7916b1..c22dfe7cea9 100644 --- a/docs/dyn/aiplatform_v1beta1.projects.locations.html +++ b/docs/dyn/aiplatform_v1beta1.projects.locations.html @@ -1917,8 +1917,17 @@Method Details
}, "metricResults": [ # Metric results for each instance. The order of the metric results is guaranteed to be the same as the order of the instances in the request. { # Result for a single metric on a single instance. - "explanation": "A String", # The explanation for the metric result. - "rubricVerdicts": [ # For rubric-based metrics, the verdicts for each rubric. + "error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Output only. The error status for the metric result. + "code": 42, # The status code, which should be an enum value of google.rpc.Code. + "details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use. + { + "a_key": "", # Properties of the object. Contains field @type with type URL. + }, + ], + "message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. + }, + "explanation": "A String", # Output only. The explanation for the metric result. + "rubricVerdicts": [ # Output only. For rubric-based metrics, the verdicts for each rubric. { # Represents the verdict of an evaluation against a single rubric. "evaluatedRubric": { # Message representing a single testable criterion for evaluation. One input prompt could have multiple rubrics. # Required. The full rubric definition that was evaluated. Storing this ensures the verdict is self-contained and understandable, especially if the original rubric definition changes or was dynamically generated. "content": { # Content of the rubric, defining the testable criteria. # Required. The actual testable criteria for the rubric. @@ -1934,7 +1943,7 @@Method Details
"verdict": True or False, # Required. Outcome of the evaluation against the rubric, represented as a boolean. `true` indicates a "Pass", `false` indicates a "Fail". }, ], - "score": 3.14, # The score for the metric. Please refer to each metric's documentation for the meaning of the score. + "score": 3.14, # Output only. The score for the metric. Please refer to each metric's documentation for the meaning of the score. }, ], "metricxResult": { # Spec for MetricX result - calculates the MetricX score for the given instance using the version specified in the spec. # Result for Metricx metric. diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.html b/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.html index df4cf7fecef..fc80c1fba26 100644 --- a/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.html +++ b/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.html @@ -247,6 +247,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". @@ -485,6 +488,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". @@ -665,6 +671,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". @@ -850,6 +859,9 @@Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created. }, "etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens. + "labels": { # Labels for the ReasoningEngine. + "a_key": "A String", + }, "name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` "spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine "agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom". diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.memories.html b/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.memories.html index b3300c825e0..0088c38407d 100644 --- a/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.memories.html +++ b/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.memories.html @@ -95,7 +95,7 @@Instance Methods
get(name, x__xgafv=None)Get a Memory.
-
+list(parent, filter=None, pageSize=None, pageToken=None, x__xgafv=None)list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)List Memories.
@@ -125,7 +125,7 @@Method Details
"createTime": "A String", # Output only. Timestamp when this Memory was created. "description": "A String", # Optional. Description of the Memory. "displayName": "A String", # Optional. Display name of the Memory. - "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output, regardless of what `expiration` was sent on input. + "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output when `expiration` is set on input, regardless of whether `expire_time` or `ttl` was provided. "fact": "A String", # Required. Semantic knowledge extracted from the source content. "name": "A String", # Identifier. The resource name of the Memory. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}/memories/{memory}` "scope": { # Required. Immutable. The scope of the Memory. Memories are isolated within their scope. The scope is defined when creating or generating memories. Scope values cannot contain the wildcard character '*'. @@ -327,7 +327,7 @@Method Details
"createTime": "A String", # Output only. Timestamp when this Memory was created. "description": "A String", # Optional. Description of the Memory. "displayName": "A String", # Optional. Display name of the Memory. - "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output, regardless of what `expiration` was sent on input. + "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output when `expiration` is set on input, regardless of whether `expire_time` or `ttl` was provided. "fact": "A String", # Required. Semantic knowledge extracted from the source content. "name": "A String", # Identifier. The resource name of the Memory. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}/memories/{memory}` "scope": { # Required. Immutable. The scope of the Memory. Memories are isolated within their scope. The scope is defined when creating or generating memories. Scope values cannot contain the wildcard character '*'. @@ -339,12 +339,13 @@Method Details
list(parent, filter=None, pageSize=None, pageToken=None, x__xgafv=None)
+ list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
List Memories.
Args:
parent: string, Required. The resource name of the ReasoningEngine to list the Memories under. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` (required)
filter: string, Optional. The standard list filter. More detail in [AIP-160](https://google.aip.dev/160). Supported fields (equality match only): * `scope` (as a JSON string)
+ orderBy: string, Optional. The standard list order by string. If not specified, the default order is `create_time desc`. If specified, the default sorting order of provided fields is ascending. More detail in [AIP-132](https://google.aip.dev/132). Supported fields: * `create_time` * `update_time`
pageSize: integer, Optional. The standard list page size.
pageToken: string, Optional. The standard list page token.
x__xgafv: string, V1 error format.
@@ -361,7 +362,7 @@ Method Details
"createTime": "A String", # Output only. Timestamp when this Memory was created.
"description": "A String", # Optional. Description of the Memory.
"displayName": "A String", # Optional. Display name of the Memory.
- "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output, regardless of what `expiration` was sent on input.
+ "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output when `expiration` is set on input, regardless of whether `expire_time` or `ttl` was provided.
"fact": "A String", # Required. Semantic knowledge extracted from the source content.
"name": "A String", # Identifier. The resource name of the Memory. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}/memories/{memory}`
"scope": { # Required. Immutable. The scope of the Memory. Memories are isolated within their scope. The scope is defined when creating or generating memories. Scope values cannot contain the wildcard character '*'.
@@ -402,7 +403,7 @@ Method Details
"createTime": "A String", # Output only. Timestamp when this Memory was created.
"description": "A String", # Optional. Description of the Memory.
"displayName": "A String", # Optional. Display name of the Memory.
- "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output, regardless of what `expiration` was sent on input.
+ "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output when `expiration` is set on input, regardless of whether `expire_time` or `ttl` was provided.
"fact": "A String", # Required. Semantic knowledge extracted from the source content.
"name": "A String", # Identifier. The resource name of the Memory. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}/memories/{memory}`
"scope": { # Required. Immutable. The scope of the Memory. Memories are isolated within their scope. The scope is defined when creating or generating memories. Scope values cannot contain the wildcard character '*'.
@@ -482,7 +483,7 @@ Method Details
"createTime": "A String", # Output only. Timestamp when this Memory was created.
"description": "A String", # Optional. Description of the Memory.
"displayName": "A String", # Optional. Display name of the Memory.
- "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output, regardless of what `expiration` was sent on input.
+ "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output when `expiration` is set on input, regardless of whether `expire_time` or `ttl` was provided.
"fact": "A String", # Required. Semantic knowledge extracted from the source content.
"name": "A String", # Identifier. The resource name of the Memory. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}/memories/{memory}`
"scope": { # Required. Immutable. The scope of the Memory. Memories are isolated within their scope. The scope is defined when creating or generating memories. Scope values cannot contain the wildcard character '*'.
diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.sandboxEnvironments.html b/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.sandboxEnvironments.html
index bfb55a75d79..35dfa1d353d 100644
--- a/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.sandboxEnvironments.html
+++ b/docs/dyn/aiplatform_v1beta1.projects.locations.reasoningEngines.sandboxEnvironments.html
@@ -118,20 +118,10 @@ Method Details
{ # SandboxEnvironment is a containerized environment that provides a customizable secure execution runtime for AI agents.
"createTime": "A String", # Output only. The timestamp when this SandboxEnvironment was created.
"displayName": "A String", # Required. The display name of the SandboxEnvironment.
- "metadata": "", # Output only. Additional information about the SandboxEnvironment.
"name": "A String", # Identifier. The name of the SandboxEnvironment.
"spec": { # The specification of a SandboxEnvironment. # Optional. The configuration of the SandboxEnvironment.
"codeExecutionEnvironment": { # The code execution environment with customized settings. # Optional. The code execution environment.
"codeLanguage": "A String", # The coding language supported in this environment.
- "dependencies": [ # Optional. The additional dependencies to install in the code execution environment. For example, "pandas==2.2.3".
- "A String",
- ],
- "env": [ # Optional. The environment variables to set in the code execution environment.
- { # Represents an environment variable present in a Container or Python Module.
- "name": "A String", # Required. Name of the environment variable. Must be a valid C identifier.
- "value": "A String", # Required. Variables that reference a $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.
- },
- ],
"machineConfig": "A String", # The machine config of the code execution environment.
},
},
@@ -266,20 +256,10 @@ Method Details
{ # SandboxEnvironment is a containerized environment that provides a customizable secure execution runtime for AI agents.
"createTime": "A String", # Output only. The timestamp when this SandboxEnvironment was created.
"displayName": "A String", # Required. The display name of the SandboxEnvironment.
- "metadata": "", # Output only. Additional information about the SandboxEnvironment.
"name": "A String", # Identifier. The name of the SandboxEnvironment.
"spec": { # The specification of a SandboxEnvironment. # Optional. The configuration of the SandboxEnvironment.
"codeExecutionEnvironment": { # The code execution environment with customized settings. # Optional. The code execution environment.
"codeLanguage": "A String", # The coding language supported in this environment.
- "dependencies": [ # Optional. The additional dependencies to install in the code execution environment. For example, "pandas==2.2.3".
- "A String",
- ],
- "env": [ # Optional. The environment variables to set in the code execution environment.
- { # Represents an environment variable present in a Container or Python Module.
- "name": "A String", # Required. Name of the environment variable. Must be a valid C identifier.
- "value": "A String", # Required. Variables that reference a $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.
- },
- ],
"machineConfig": "A String", # The machine config of the code execution environment.
},
},
@@ -311,20 +291,10 @@ Method Details
{ # SandboxEnvironment is a containerized environment that provides a customizable secure execution runtime for AI agents.
"createTime": "A String", # Output only. The timestamp when this SandboxEnvironment was created.
"displayName": "A String", # Required. The display name of the SandboxEnvironment.
- "metadata": "", # Output only. Additional information about the SandboxEnvironment.
"name": "A String", # Identifier. The name of the SandboxEnvironment.
"spec": { # The specification of a SandboxEnvironment. # Optional. The configuration of the SandboxEnvironment.
"codeExecutionEnvironment": { # The code execution environment with customized settings. # Optional. The code execution environment.
"codeLanguage": "A String", # The coding language supported in this environment.
- "dependencies": [ # Optional. The additional dependencies to install in the code execution environment. For example, "pandas==2.2.3".
- "A String",
- ],
- "env": [ # Optional. The environment variables to set in the code execution environment.
- { # Represents an environment variable present in a Container or Python Module.
- "name": "A String", # Required. Name of the environment variable. Must be a valid C identifier.
- "value": "A String", # Required. Variables that reference a $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.
- },
- ],
"machineConfig": "A String", # The machine config of the code execution environment.
},
},
diff --git a/docs/dyn/aiplatform_v1beta1.projects.locations.tuningJobs.html b/docs/dyn/aiplatform_v1beta1.projects.locations.tuningJobs.html
index cf3119ce0a5..5af15df7597 100644
--- a/docs/dyn/aiplatform_v1beta1.projects.locations.tuningJobs.html
+++ b/docs/dyn/aiplatform_v1beta1.projects.locations.tuningJobs.html
@@ -312,6 +312,7 @@ Method Details
},
},
},
+ "exportLastCheckpointOnly": True or False, # Optional. If set to true, disable intermediate checkpoints for Preference Optimization and only the last checkpoint will be exported. Otherwise, enable intermediate checkpoints for Preference Optimization. Default is false.
"hyperParameters": { # Hyperparameters for Preference Optimization. # Optional. Hyperparameters for Preference Optimization.
"adapterSize": "A String", # Optional. Adapter size for preference optimization.
"beta": 3.14, # Optional. Weight for KL Divergence regularization.
@@ -1001,6 +1002,7 @@ Method Details
},
},
},
+ "exportLastCheckpointOnly": True or False, # Optional. If set to true, disable intermediate checkpoints for Preference Optimization and only the last checkpoint will be exported. Otherwise, enable intermediate checkpoints for Preference Optimization. Default is false.
"hyperParameters": { # Hyperparameters for Preference Optimization. # Optional. Hyperparameters for Preference Optimization.
"adapterSize": "A String", # Optional. Adapter size for preference optimization.
"beta": 3.14, # Optional. Weight for KL Divergence regularization.
@@ -1697,6 +1699,7 @@ Method Details
},
},
},
+ "exportLastCheckpointOnly": True or False, # Optional. If set to true, disable intermediate checkpoints for Preference Optimization and only the last checkpoint will be exported. Otherwise, enable intermediate checkpoints for Preference Optimization. Default is false.
"hyperParameters": { # Hyperparameters for Preference Optimization. # Optional. Hyperparameters for Preference Optimization.
"adapterSize": "A String", # Optional. Adapter size for preference optimization.
"beta": 3.14, # Optional. Weight for KL Divergence regularization.
@@ -2399,6 +2402,7 @@ Method Details
},
},
},
+ "exportLastCheckpointOnly": True or False, # Optional. If set to true, disable intermediate checkpoints for Preference Optimization and only the last checkpoint will be exported. Otherwise, enable intermediate checkpoints for Preference Optimization. Default is false.
"hyperParameters": { # Hyperparameters for Preference Optimization. # Optional. Hyperparameters for Preference Optimization.
"adapterSize": "A String", # Optional. Adapter size for preference optimization.
"beta": 3.14, # Optional. Weight for KL Divergence regularization.
@@ -3234,6 +3238,7 @@ Method Details
},
},
},
+ "exportLastCheckpointOnly": True or False, # Optional. If set to true, disable intermediate checkpoints for Preference Optimization and only the last checkpoint will be exported. Otherwise, enable intermediate checkpoints for Preference Optimization. Default is false.
"hyperParameters": { # Hyperparameters for Preference Optimization. # Optional. Hyperparameters for Preference Optimization.
"adapterSize": "A String", # Optional. Adapter size for preference optimization.
"beta": 3.14, # Optional. Weight for KL Divergence regularization.
diff --git a/docs/dyn/aiplatform_v1beta1.reasoningEngines.html b/docs/dyn/aiplatform_v1beta1.reasoningEngines.html
index 5991fa31b52..75fead300de 100644
--- a/docs/dyn/aiplatform_v1beta1.reasoningEngines.html
+++ b/docs/dyn/aiplatform_v1beta1.reasoningEngines.html
@@ -231,6 +231,9 @@ Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created.
},
"etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
+ "labels": { # Labels for the ReasoningEngine.
+ "a_key": "A String",
+ },
"name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}`
"spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine
"agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom".
@@ -470,6 +473,9 @@ Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created.
},
"etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
+ "labels": { # Labels for the ReasoningEngine.
+ "a_key": "A String",
+ },
"name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}`
"spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine
"agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom".
@@ -650,6 +656,9 @@ Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created.
},
"etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
+ "labels": { # Labels for the ReasoningEngine.
+ "a_key": "A String",
+ },
"name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}`
"spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine
"agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom".
@@ -835,6 +844,9 @@ Method Details
"kmsKeyName": "A String", # Required. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`. The key needs to be in the same region as where the compute resource is created.
},
"etag": "A String", # Optional. Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
+ "labels": { # Labels for the ReasoningEngine.
+ "a_key": "A String",
+ },
"name": "A String", # Identifier. The resource name of the ReasoningEngine. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}`
"spec": { # ReasoningEngine configurations # Optional. Configurations of the ReasoningEngine
"agentFramework": "A String", # Optional. The OSS agent framework used to develop the agent. Currently supported values: "google-adk", "langchain", "langgraph", "ag2", "llama-index", "custom".
diff --git a/docs/dyn/aiplatform_v1beta1.reasoningEngines.memories.html b/docs/dyn/aiplatform_v1beta1.reasoningEngines.memories.html
index 64fa4e179a8..f7d53e66596 100644
--- a/docs/dyn/aiplatform_v1beta1.reasoningEngines.memories.html
+++ b/docs/dyn/aiplatform_v1beta1.reasoningEngines.memories.html
@@ -90,7 +90,7 @@ Instance Methods
get(name, x__xgafv=None)
Get a Memory.
- list(parent, filter=None, pageSize=None, pageToken=None, x__xgafv=None)
+ list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
List Memories.
@@ -120,7 +120,7 @@ Method Details
"createTime": "A String", # Output only. Timestamp when this Memory was created.
"description": "A String", # Optional. Description of the Memory.
"displayName": "A String", # Optional. Display name of the Memory.
- "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output, regardless of what `expiration` was sent on input.
+ "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output when `expiration` is set on input, regardless of whether `expire_time` or `ttl` was provided.
"fact": "A String", # Required. Semantic knowledge extracted from the source content.
"name": "A String", # Identifier. The resource name of the Memory. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}/memories/{memory}`
"scope": { # Required. Immutable. The scope of the Memory. Memories are isolated within their scope. The scope is defined when creating or generating memories. Scope values cannot contain the wildcard character '*'.
@@ -322,7 +322,7 @@ Method Details
"createTime": "A String", # Output only. Timestamp when this Memory was created.
"description": "A String", # Optional. Description of the Memory.
"displayName": "A String", # Optional. Display name of the Memory.
- "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output, regardless of what `expiration` was sent on input.
+ "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output when `expiration` is set on input, regardless of whether `expire_time` or `ttl` was provided.
"fact": "A String", # Required. Semantic knowledge extracted from the source content.
"name": "A String", # Identifier. The resource name of the Memory. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}/memories/{memory}`
"scope": { # Required. Immutable. The scope of the Memory. Memories are isolated within their scope. The scope is defined when creating or generating memories. Scope values cannot contain the wildcard character '*'.
@@ -334,12 +334,13 @@ Method Details
list(parent, filter=None, pageSize=None, pageToken=None, x__xgafv=None)
+ list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
List Memories.
Args:
parent: string, Required. The resource name of the ReasoningEngine to list the Memories under. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}` (required)
filter: string, Optional. The standard list filter. More detail in [AIP-160](https://google.aip.dev/160). Supported fields (equality match only): * `scope` (as a JSON string)
+ orderBy: string, Optional. The standard list order by string. If not specified, the default order is `create_time desc`. If specified, the default sorting order of provided fields is ascending. More detail in [AIP-132](https://google.aip.dev/132). Supported fields: * `create_time` * `update_time`
pageSize: integer, Optional. The standard list page size.
pageToken: string, Optional. The standard list page token.
x__xgafv: string, V1 error format.
@@ -356,7 +357,7 @@ Method Details
"createTime": "A String", # Output only. Timestamp when this Memory was created.
"description": "A String", # Optional. Description of the Memory.
"displayName": "A String", # Optional. Display name of the Memory.
- "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output, regardless of what `expiration` was sent on input.
+ "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output when `expiration` is set on input, regardless of whether `expire_time` or `ttl` was provided.
"fact": "A String", # Required. Semantic knowledge extracted from the source content.
"name": "A String", # Identifier. The resource name of the Memory. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}/memories/{memory}`
"scope": { # Required. Immutable. The scope of the Memory. Memories are isolated within their scope. The scope is defined when creating or generating memories. Scope values cannot contain the wildcard character '*'.
@@ -397,7 +398,7 @@ Method Details
"createTime": "A String", # Output only. Timestamp when this Memory was created.
"description": "A String", # Optional. Description of the Memory.
"displayName": "A String", # Optional. Display name of the Memory.
- "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output, regardless of what `expiration` was sent on input.
+ "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output when `expiration` is set on input, regardless of whether `expire_time` or `ttl` was provided.
"fact": "A String", # Required. Semantic knowledge extracted from the source content.
"name": "A String", # Identifier. The resource name of the Memory. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}/memories/{memory}`
"scope": { # Required. Immutable. The scope of the Memory. Memories are isolated within their scope. The scope is defined when creating or generating memories. Scope values cannot contain the wildcard character '*'.
@@ -477,7 +478,7 @@ Method Details
"createTime": "A String", # Output only. Timestamp when this Memory was created.
"description": "A String", # Optional. Description of the Memory.
"displayName": "A String", # Optional. Display name of the Memory.
- "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output, regardless of what `expiration` was sent on input.
+ "expireTime": "A String", # Optional. Timestamp of when this resource is considered expired. This is *always* provided on output when `expiration` is set on input, regardless of whether `expire_time` or `ttl` was provided.
"fact": "A String", # Required. Semantic knowledge extracted from the source content.
"name": "A String", # Identifier. The resource name of the Memory. Format: `projects/{project}/locations/{location}/reasoningEngines/{reasoning_engine}/memories/{memory}`
"scope": { # Required. Immutable. The scope of the Memory. Memories are isolated within their scope. The scope is defined when creating or generating memories. Scope values cannot contain the wildcard character '*'.
diff --git a/docs/dyn/alloydb_v1.projects.locations.html b/docs/dyn/alloydb_v1.projects.locations.html
index 84a81a23edd..163bb541f0a 100644
--- a/docs/dyn/alloydb_v1.projects.locations.html
+++ b/docs/dyn/alloydb_v1.projects.locations.html
@@ -145,7 +145,7 @@ Method Details
Args:
name: string, The resource that owns the locations collection, if applicable. (required)
- extraLocationTypes: string, Optional. Do not use this field. It is unsupported and is ignored unless explicitly documented otherwise. This is primarily for internal usage. (repeated)
+ extraLocationTypes: string, Optional. Unless explicitly documented otherwise, don't use this unsupported field which is primarily intended for internal usage. (repeated)
filter: string, A filter to narrow down results to a preferred subset. The filtering language accepts strings like `"displayName=tokyo"`, and is documented in more detail in [AIP-160](https://google.aip.dev/160).
pageSize: integer, The maximum number of results to return. If not set, the service selects a default.
pageToken: string, A page token received from the `next_page_token` field in the response. Send that page token to receive the subsequent page.
diff --git a/docs/dyn/alloydb_v1alpha.projects.locations.html b/docs/dyn/alloydb_v1alpha.projects.locations.html
index 2234500c30b..f995e6090b8 100644
--- a/docs/dyn/alloydb_v1alpha.projects.locations.html
+++ b/docs/dyn/alloydb_v1alpha.projects.locations.html
@@ -145,7 +145,7 @@ Method Details
Args:
name: string, The resource that owns the locations collection, if applicable. (required)
- extraLocationTypes: string, Optional. Do not use this field. It is unsupported and is ignored unless explicitly documented otherwise. This is primarily for internal usage. (repeated)
+ extraLocationTypes: string, Optional. Unless explicitly documented otherwise, don't use this unsupported field which is primarily intended for internal usage. (repeated)
filter: string, A filter to narrow down results to a preferred subset. The filtering language accepts strings like `"displayName=tokyo"`, and is documented in more detail in [AIP-160](https://google.aip.dev/160).
pageSize: integer, The maximum number of results to return. If not set, the service selects a default.
pageToken: string, A page token received from the `next_page_token` field in the response. Send that page token to receive the subsequent page.
diff --git a/docs/dyn/alloydb_v1beta.projects.locations.html b/docs/dyn/alloydb_v1beta.projects.locations.html
index 7a5ea1573b9..6979f2b817b 100644
--- a/docs/dyn/alloydb_v1beta.projects.locations.html
+++ b/docs/dyn/alloydb_v1beta.projects.locations.html
@@ -145,7 +145,7 @@ Method Details
Args:
name: string, The resource that owns the locations collection, if applicable. (required)
- extraLocationTypes: string, Optional. Do not use this field. It is unsupported and is ignored unless explicitly documented otherwise. This is primarily for internal usage. (repeated)
+ extraLocationTypes: string, Optional. Unless explicitly documented otherwise, don't use this unsupported field which is primarily intended for internal usage. (repeated)
filter: string, A filter to narrow down results to a preferred subset. The filtering language accepts strings like `"displayName=tokyo"`, and is documented in more detail in [AIP-160](https://google.aip.dev/160).
pageSize: integer, The maximum number of results to return. If not set, the service selects a default.
pageToken: string, A page token received from the `next_page_token` field in the response. Send that page token to receive the subsequent page.
diff --git a/docs/dyn/analyticshub_v1.projects.locations.dataExchanges.html b/docs/dyn/analyticshub_v1.projects.locations.dataExchanges.html
index 0a6132b6bed..bbd430460bd 100644
--- a/docs/dyn/analyticshub_v1.projects.locations.dataExchanges.html
+++ b/docs/dyn/analyticshub_v1.projects.locations.dataExchanges.html
@@ -378,6 +378,9 @@ Method Details
"a_key": "A String",
},
"location": "A String", # Required. The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
+ "replicaLocations": [ # Optional. The geographic locations where the dataset should be replicated. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations.
+ "A String",
+ ],
},
"lastModifyTime": "A String", # Output only. Timestamp when the subscription was last modified.
"linkedDatasetMap": { # Output only. Map of listing resource names to associated linked resource, e.g. projects/123/locations/us/dataExchanges/456/listings/789 -> projects/123/datasets/my_dataset For listing-level subscriptions, this is a map of size 1. Only contains values if state == STATE_ACTIVE.
@@ -601,6 +604,9 @@ Method Details
"a_key": "A String",
},
"location": "A String", # Required. The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
+ "replicaLocations": [ # Optional. The geographic locations where the dataset should be replicated. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations.
+ "A String",
+ ],
},
"subscriberContact": "A String", # Email of the subscriber.
"subscription": "A String", # Required. Name of the subscription to create. e.g. `subscription1`
diff --git a/docs/dyn/analyticshub_v1.projects.locations.dataExchanges.listings.html b/docs/dyn/analyticshub_v1.projects.locations.dataExchanges.listings.html
index 9e1f3348b26..ef329a79139 100644
--- a/docs/dyn/analyticshub_v1.projects.locations.dataExchanges.listings.html
+++ b/docs/dyn/analyticshub_v1.projects.locations.dataExchanges.listings.html
@@ -132,6 +132,16 @@ Method Details
"allowOnlyMetadataSharing": True or False, # Optional. If true, the listing is only available to get the resource metadata. Listing is non subscribable.
"bigqueryDataset": { # A reference to a shared dataset. It is an existing BigQuery dataset with a collection of objects such as tables and views that you want to share with subscribers. When subscriber's subscribe to a listing, Analytics Hub creates a linked dataset in the subscriber's project. A Linked dataset is an opaque, read-only BigQuery dataset that serves as a _symbolic link_ to a shared dataset. # Shared dataset i.e. BigQuery dataset source.
"dataset": "A String", # Optional. Resource name of the dataset source for this listing. e.g. `projects/myproject/datasets/123`
+ "effectiveReplicas": [ # Output only. Server-owned effective state of replicas. Contains both primary and secondary replicas. Each replica includes a system-computed (output-only) state and primary designation.
+ { # Represents the state of a replica of a shared dataset. It includes the geographic location of the replica and system-computed, output-only fields indicating its replication state and whether it is the primary replica.
+ "location": "A String", # Output only. The geographic location where the replica resides. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations. Eg. "us-central1".
+ "primaryState": "A String", # Output only. Indicates that this replica is the primary replica.
+ "replicaState": "A String", # Output only. Assigned by Analytics Hub based on real BigQuery replication state.
+ },
+ ],
+ "replicaLocations": [ # Optional. A list of regions where the publisher has created shared dataset replicas.
+ "A String",
+ ],
"restrictedExportPolicy": { # Restricted export policy used to configure restricted export on linked dataset. # Optional. If set, restricted export policy will be propagated and enforced on the linked dataset.
"enabled": True or False, # Optional. If true, enable restricted export.
"restrictDirectTableAccess": True or False, # Optional. If true, restrict direct table access (read api/tabledata.list) on linked table.
@@ -198,6 +208,16 @@ Method Details
"allowOnlyMetadataSharing": True or False, # Optional. If true, the listing is only available to get the resource metadata. Listing is non subscribable.
"bigqueryDataset": { # A reference to a shared dataset. It is an existing BigQuery dataset with a collection of objects such as tables and views that you want to share with subscribers. When subscriber's subscribe to a listing, Analytics Hub creates a linked dataset in the subscriber's project. A Linked dataset is an opaque, read-only BigQuery dataset that serves as a _symbolic link_ to a shared dataset. # Shared dataset i.e. BigQuery dataset source.
"dataset": "A String", # Optional. Resource name of the dataset source for this listing. e.g. `projects/myproject/datasets/123`
+ "effectiveReplicas": [ # Output only. Server-owned effective state of replicas. Contains both primary and secondary replicas. Each replica includes a system-computed (output-only) state and primary designation.
+ { # Represents the state of a replica of a shared dataset. It includes the geographic location of the replica and system-computed, output-only fields indicating its replication state and whether it is the primary replica.
+ "location": "A String", # Output only. The geographic location where the replica resides. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations. Eg. "us-central1".
+ "primaryState": "A String", # Output only. Indicates that this replica is the primary replica.
+ "replicaState": "A String", # Output only. Assigned by Analytics Hub based on real BigQuery replication state.
+ },
+ ],
+ "replicaLocations": [ # Optional. A list of regions where the publisher has created shared dataset replicas.
+ "A String",
+ ],
"restrictedExportPolicy": { # Restricted export policy used to configure restricted export on linked dataset. # Optional. If set, restricted export policy will be propagated and enforced on the linked dataset.
"enabled": True or False, # Optional. If true, enable restricted export.
"restrictDirectTableAccess": True or False, # Optional. If true, restrict direct table access (read api/tabledata.list) on linked table.
@@ -289,6 +309,16 @@ Method Details
"allowOnlyMetadataSharing": True or False, # Optional. If true, the listing is only available to get the resource metadata. Listing is non subscribable.
"bigqueryDataset": { # A reference to a shared dataset. It is an existing BigQuery dataset with a collection of objects such as tables and views that you want to share with subscribers. When subscriber's subscribe to a listing, Analytics Hub creates a linked dataset in the subscriber's project. A Linked dataset is an opaque, read-only BigQuery dataset that serves as a _symbolic link_ to a shared dataset. # Shared dataset i.e. BigQuery dataset source.
"dataset": "A String", # Optional. Resource name of the dataset source for this listing. e.g. `projects/myproject/datasets/123`
+ "effectiveReplicas": [ # Output only. Server-owned effective state of replicas. Contains both primary and secondary replicas. Each replica includes a system-computed (output-only) state and primary designation.
+ { # Represents the state of a replica of a shared dataset. It includes the geographic location of the replica and system-computed, output-only fields indicating its replication state and whether it is the primary replica.
+ "location": "A String", # Output only. The geographic location where the replica resides. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations. Eg. "us-central1".
+ "primaryState": "A String", # Output only. Indicates that this replica is the primary replica.
+ "replicaState": "A String", # Output only. Assigned by Analytics Hub based on real BigQuery replication state.
+ },
+ ],
+ "replicaLocations": [ # Optional. A list of regions where the publisher has created shared dataset replicas.
+ "A String",
+ ],
"restrictedExportPolicy": { # Restricted export policy used to configure restricted export on linked dataset. # Optional. If set, restricted export policy will be propagated and enforced on the linked dataset.
"enabled": True or False, # Optional. If true, enable restricted export.
"restrictDirectTableAccess": True or False, # Optional. If true, restrict direct table access (read api/tabledata.list) on linked table.
@@ -421,6 +451,16 @@ Method Details
"allowOnlyMetadataSharing": True or False, # Optional. If true, the listing is only available to get the resource metadata. Listing is non subscribable.
"bigqueryDataset": { # A reference to a shared dataset. It is an existing BigQuery dataset with a collection of objects such as tables and views that you want to share with subscribers. When subscriber's subscribe to a listing, Analytics Hub creates a linked dataset in the subscriber's project. A Linked dataset is an opaque, read-only BigQuery dataset that serves as a _symbolic link_ to a shared dataset. # Shared dataset i.e. BigQuery dataset source.
"dataset": "A String", # Optional. Resource name of the dataset source for this listing. e.g. `projects/myproject/datasets/123`
+ "effectiveReplicas": [ # Output only. Server-owned effective state of replicas. Contains both primary and secondary replicas. Each replica includes a system-computed (output-only) state and primary designation.
+ { # Represents the state of a replica of a shared dataset. It includes the geographic location of the replica and system-computed, output-only fields indicating its replication state and whether it is the primary replica.
+ "location": "A String", # Output only. The geographic location where the replica resides. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations. Eg. "us-central1".
+ "primaryState": "A String", # Output only. Indicates that this replica is the primary replica.
+ "replicaState": "A String", # Output only. Assigned by Analytics Hub based on real BigQuery replication state.
+ },
+ ],
+ "replicaLocations": [ # Optional. A list of regions where the publisher has created shared dataset replicas.
+ "A String",
+ ],
"restrictedExportPolicy": { # Restricted export policy used to configure restricted export on linked dataset. # Optional. If set, restricted export policy will be propagated and enforced on the linked dataset.
"enabled": True or False, # Optional. If true, enable restricted export.
"restrictDirectTableAccess": True or False, # Optional. If true, restrict direct table access (read api/tabledata.list) on linked table.
@@ -517,6 +557,9 @@ Method Details
"a_key": "A String",
},
"location": "A String", # Required. The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
+ "replicaLocations": [ # Optional. The geographic locations where the dataset should be replicated. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations.
+ "A String",
+ ],
},
"lastModifyTime": "A String", # Output only. Timestamp when the subscription was last modified.
"linkedDatasetMap": { # Output only. Map of listing resource names to associated linked resource, e.g. projects/123/locations/us/dataExchanges/456/listings/789 -> projects/123/datasets/my_dataset For listing-level subscriptions, this is a map of size 1. Only contains values if state == STATE_ACTIVE.
@@ -587,6 +630,16 @@ Method Details
"allowOnlyMetadataSharing": True or False, # Optional. If true, the listing is only available to get the resource metadata. Listing is non subscribable.
"bigqueryDataset": { # A reference to a shared dataset. It is an existing BigQuery dataset with a collection of objects such as tables and views that you want to share with subscribers. When subscriber's subscribe to a listing, Analytics Hub creates a linked dataset in the subscriber's project. A Linked dataset is an opaque, read-only BigQuery dataset that serves as a _symbolic link_ to a shared dataset. # Shared dataset i.e. BigQuery dataset source.
"dataset": "A String", # Optional. Resource name of the dataset source for this listing. e.g. `projects/myproject/datasets/123`
+ "effectiveReplicas": [ # Output only. Server-owned effective state of replicas. Contains both primary and secondary replicas. Each replica includes a system-computed (output-only) state and primary designation.
+ { # Represents the state of a replica of a shared dataset. It includes the geographic location of the replica and system-computed, output-only fields indicating its replication state and whether it is the primary replica.
+ "location": "A String", # Output only. The geographic location where the replica resides. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations. Eg. "us-central1".
+ "primaryState": "A String", # Output only. Indicates that this replica is the primary replica.
+ "replicaState": "A String", # Output only. Assigned by Analytics Hub based on real BigQuery replication state.
+ },
+ ],
+ "replicaLocations": [ # Optional. A list of regions where the publisher has created shared dataset replicas.
+ "A String",
+ ],
"restrictedExportPolicy": { # Restricted export policy used to configure restricted export on linked dataset. # Optional. If set, restricted export policy will be propagated and enforced on the linked dataset.
"enabled": True or False, # Optional. If true, enable restricted export.
"restrictDirectTableAccess": True or False, # Optional. If true, restrict direct table access (read api/tabledata.list) on linked table.
@@ -653,6 +706,16 @@ Method Details
"allowOnlyMetadataSharing": True or False, # Optional. If true, the listing is only available to get the resource metadata. Listing is non subscribable.
"bigqueryDataset": { # A reference to a shared dataset. It is an existing BigQuery dataset with a collection of objects such as tables and views that you want to share with subscribers. When subscriber's subscribe to a listing, Analytics Hub creates a linked dataset in the subscriber's project. A Linked dataset is an opaque, read-only BigQuery dataset that serves as a _symbolic link_ to a shared dataset. # Shared dataset i.e. BigQuery dataset source.
"dataset": "A String", # Optional. Resource name of the dataset source for this listing. e.g. `projects/myproject/datasets/123`
+ "effectiveReplicas": [ # Output only. Server-owned effective state of replicas. Contains both primary and secondary replicas. Each replica includes a system-computed (output-only) state and primary designation.
+ { # Represents the state of a replica of a shared dataset. It includes the geographic location of the replica and system-computed, output-only fields indicating its replication state and whether it is the primary replica.
+ "location": "A String", # Output only. The geographic location where the replica resides. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations. Eg. "us-central1".
+ "primaryState": "A String", # Output only. Indicates that this replica is the primary replica.
+ "replicaState": "A String", # Output only. Assigned by Analytics Hub based on real BigQuery replication state.
+ },
+ ],
+ "replicaLocations": [ # Optional. A list of regions where the publisher has created shared dataset replicas.
+ "A String",
+ ],
"restrictedExportPolicy": { # Restricted export policy used to configure restricted export on linked dataset. # Optional. If set, restricted export policy will be propagated and enforced on the linked dataset.
"enabled": True or False, # Optional. If true, enable restricted export.
"restrictDirectTableAccess": True or False, # Optional. If true, restrict direct table access (read api/tabledata.list) on linked table.
@@ -813,6 +876,9 @@ Method Details
"a_key": "A String",
},
"location": "A String", # Required. The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
+ "replicaLocations": [ # Optional. The geographic locations where the dataset should be replicated. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations.
+ "A String",
+ ],
},
"destinationPubsubSubscription": { # Defines the destination Pub/Sub subscription. # Input only. Destination Pub/Sub subscription to create for the subscriber.
"pubsubSubscription": { # Defines the destination Pub/Sub subscription. If none of `push_config`, `bigquery_config`, `cloud_storage_config`, `pubsub_export_config`, or `pubsublite_export_config` is set, then the subscriber will pull and ack messages using API methods. At most one of these fields may be set. # Required. Destination Pub/Sub subscription resource.
@@ -919,6 +985,9 @@ Method Details
"a_key": "A String",
},
"location": "A String", # Required. The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
+ "replicaLocations": [ # Optional. The geographic locations where the dataset should be replicated. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations.
+ "A String",
+ ],
},
"lastModifyTime": "A String", # Output only. Timestamp when the subscription was last modified.
"linkedDatasetMap": { # Output only. Map of listing resource names to associated linked resource, e.g. projects/123/locations/us/dataExchanges/456/listings/789 -> projects/123/datasets/my_dataset For listing-level subscriptions, this is a map of size 1. Only contains values if state == STATE_ACTIVE.
diff --git a/docs/dyn/analyticshub_v1.projects.locations.subscriptions.html b/docs/dyn/analyticshub_v1.projects.locations.subscriptions.html
index 8d41798fdd8..294515338a6 100644
--- a/docs/dyn/analyticshub_v1.projects.locations.subscriptions.html
+++ b/docs/dyn/analyticshub_v1.projects.locations.subscriptions.html
@@ -175,6 +175,9 @@ Method Details
"a_key": "A String",
},
"location": "A String", # Required. The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
+ "replicaLocations": [ # Optional. The geographic locations where the dataset should be replicated. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations.
+ "A String",
+ ],
},
"lastModifyTime": "A String", # Output only. Timestamp when the subscription was last modified.
"linkedDatasetMap": { # Output only. Map of listing resource names to associated linked resource, e.g. projects/123/locations/us/dataExchanges/456/listings/789 -> projects/123/datasets/my_dataset For listing-level subscriptions, this is a map of size 1. Only contains values if state == STATE_ACTIVE.
@@ -297,6 +300,9 @@ Method Details
"a_key": "A String",
},
"location": "A String", # Required. The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
+ "replicaLocations": [ # Optional. The geographic locations where the dataset should be replicated. See [BigQuery locations](https://cloud.google.com/bigquery/docs/locations) for supported locations.
+ "A String",
+ ],
},
"lastModifyTime": "A String", # Output only. Timestamp when the subscription was last modified.
"linkedDatasetMap": { # Output only. Map of listing resource names to associated linked resource, e.g. projects/123/locations/us/dataExchanges/456/listings/789 -> projects/123/datasets/my_dataset For listing-level subscriptions, this is a map of size 1. Only contains values if state == STATE_ACTIVE.
diff --git a/docs/dyn/androidmanagement_v1.enterprises.devices.html b/docs/dyn/androidmanagement_v1.enterprises.devices.html
index 331de6d9977..05918af8c22 100644
--- a/docs/dyn/androidmanagement_v1.enterprises.devices.html
+++ b/docs/dyn/androidmanagement_v1.enterprises.devices.html
@@ -179,7 +179,7 @@ Method Details
"versionName": "A String", # The app version as displayed to the user.
},
],
- "appliedPasswordPolicies": [ # The password requirements currently applied to the device. The applied requirements may be slightly different from those specified in passwordPolicies in some cases. fieldPath is set based on passwordPolicies.
+ "appliedPasswordPolicies": [ # The password requirements currently applied to the device. This field exists because the applied requirements may be slightly different from those specified in passwordPolicies in some cases. Note that this field does not provide information about password compliance. For non-compliance information, see nonComplianceDetails. NonComplianceDetail.fieldPath, is set based on passwordPolicies, not based on this field.
{ # Requirements for the password used to unlock a device.
"maximumFailedPasswordsForWipe": 42, # Number of incorrect device-unlock passwords that can be entered before a device is wiped. A value of 0 means there is no restriction.
"passwordExpirationTimeout": "A String", # Password expiration timeout.
@@ -267,7 +267,7 @@ Method Details
"hardware": "A String", # Name of the hardware. For example, Angler.
"manufacturer": "A String", # Manufacturer. For example, Motorola.
"model": "A String", # The model of the device. For example, Asus Nexus 7.
- "serialNumber": "A String", # The device serial number.
+ "serialNumber": "A String", # The device serial number. However, for personally-owned devices running Android 12 and above, this is the same as the enterpriseSpecificId.
"skinShutdownTemperatures": [ # Device skin shutdown temperature thresholds in Celsius.
3.14,
],
@@ -596,7 +596,7 @@ Method Details
"versionName": "A String", # The app version as displayed to the user.
},
],
- "appliedPasswordPolicies": [ # The password requirements currently applied to the device. The applied requirements may be slightly different from those specified in passwordPolicies in some cases. fieldPath is set based on passwordPolicies.
+ "appliedPasswordPolicies": [ # The password requirements currently applied to the device. This field exists because the applied requirements may be slightly different from those specified in passwordPolicies in some cases. Note that this field does not provide information about password compliance. For non-compliance information, see nonComplianceDetails. NonComplianceDetail.fieldPath, is set based on passwordPolicies, not based on this field.
{ # Requirements for the password used to unlock a device.
"maximumFailedPasswordsForWipe": 42, # Number of incorrect device-unlock passwords that can be entered before a device is wiped. A value of 0 means there is no restriction.
"passwordExpirationTimeout": "A String", # Password expiration timeout.
@@ -684,7 +684,7 @@ Method Details
"hardware": "A String", # Name of the hardware. For example, Angler.
"manufacturer": "A String", # Manufacturer. For example, Motorola.
"model": "A String", # The model of the device. For example, Asus Nexus 7.
- "serialNumber": "A String", # The device serial number.
+ "serialNumber": "A String", # The device serial number. However, for personally-owned devices running Android 12 and above, this is the same as the enterpriseSpecificId.
"skinShutdownTemperatures": [ # Device skin shutdown temperature thresholds in Celsius.
3.14,
],
@@ -881,7 +881,7 @@ Method Details
"versionName": "A String", # The app version as displayed to the user.
},
],
- "appliedPasswordPolicies": [ # The password requirements currently applied to the device. The applied requirements may be slightly different from those specified in passwordPolicies in some cases. fieldPath is set based on passwordPolicies.
+ "appliedPasswordPolicies": [ # The password requirements currently applied to the device. This field exists because the applied requirements may be slightly different from those specified in passwordPolicies in some cases. Note that this field does not provide information about password compliance. For non-compliance information, see nonComplianceDetails. NonComplianceDetail.fieldPath, is set based on passwordPolicies, not based on this field.
{ # Requirements for the password used to unlock a device.
"maximumFailedPasswordsForWipe": 42, # Number of incorrect device-unlock passwords that can be entered before a device is wiped. A value of 0 means there is no restriction.
"passwordExpirationTimeout": "A String", # Password expiration timeout.
@@ -969,7 +969,7 @@ Method Details
"hardware": "A String", # Name of the hardware. For example, Angler.
"manufacturer": "A String", # Manufacturer. For example, Motorola.
"model": "A String", # The model of the device. For example, Asus Nexus 7.
- "serialNumber": "A String", # The device serial number.
+ "serialNumber": "A String", # The device serial number. However, for personally-owned devices running Android 12 and above, this is the same as the enterpriseSpecificId.
"skinShutdownTemperatures": [ # Device skin shutdown temperature thresholds in Celsius.
3.14,
],
@@ -1148,7 +1148,7 @@ Method Details
"versionName": "A String", # The app version as displayed to the user.
},
],
- "appliedPasswordPolicies": [ # The password requirements currently applied to the device. The applied requirements may be slightly different from those specified in passwordPolicies in some cases. fieldPath is set based on passwordPolicies.
+ "appliedPasswordPolicies": [ # The password requirements currently applied to the device. This field exists because the applied requirements may be slightly different from those specified in passwordPolicies in some cases. Note that this field does not provide information about password compliance. For non-compliance information, see nonComplianceDetails. NonComplianceDetail.fieldPath, is set based on passwordPolicies, not based on this field.
{ # Requirements for the password used to unlock a device.
"maximumFailedPasswordsForWipe": 42, # Number of incorrect device-unlock passwords that can be entered before a device is wiped. A value of 0 means there is no restriction.
"passwordExpirationTimeout": "A String", # Password expiration timeout.
@@ -1236,7 +1236,7 @@ Method Details
"hardware": "A String", # Name of the hardware. For example, Angler.
"manufacturer": "A String", # Manufacturer. For example, Motorola.
"model": "A String", # The model of the device. For example, Asus Nexus 7.
- "serialNumber": "A String", # The device serial number.
+ "serialNumber": "A String", # The device serial number. However, for personally-owned devices running Android 12 and above, this is the same as the enterpriseSpecificId.
"skinShutdownTemperatures": [ # Device skin shutdown temperature thresholds in Celsius.
3.14,
],
diff --git a/docs/dyn/androidpublisher_v3.purchases.subscriptionsv2.html b/docs/dyn/androidpublisher_v3.purchases.subscriptionsv2.html
index a999e6ebe90..aa0722c9bf4 100644
--- a/docs/dyn/androidpublisher_v3.purchases.subscriptionsv2.html
+++ b/docs/dyn/androidpublisher_v3.purchases.subscriptionsv2.html
@@ -74,6 +74,9 @@
Google Play Android Developer API . purchases . subscriptionsv2
Instance Methods
+
+ cancel(packageName, token, body=None, x__xgafv=None)
+Cancel a subscription purchase for the user.
Close httplib2 connections.
@@ -84,6 +87,34 @@ Instance Methods
revoke(packageName, token, body=None, x__xgafv=None)
Revoke a subscription purchase for the user.
Method Details
+
+ cancel(packageName, token, body=None, x__xgafv=None)
+ Cancel a subscription purchase for the user.
+
+Args:
+ packageName: string, Required. The package of the application for which this subscription was purchased (for example, 'com.some.thing'). (required)
+ token: string, Required. The token provided to the user's device when the subscription was purchased. (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # Request for the purchases.subscriptionsv2.cancel API.
+ "cancellationContext": { # Cancellation context of the purchases.subscriptionsv2.cancel API. # Required. Additional details around the subscription revocation.
+ "cancellationType": "A String", # Required. The type of cancellation for the purchased subscription.
+ },
+}
+
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # Response for the purchases.subscriptionsv2.cancel API.
+}
+
+
close()
Close httplib2 connections.
@@ -149,6 +180,15 @@ Method Details
"priceChangeMode": "A String", # Price change mode specifies how the subscription item price is changing.
"priceChangeState": "A String", # State the price change is currently in.
},
+ "priceStepUpConsentDetails": { # Information related to a price step-up that requires user consent. # The information of the latest price step-up consent.
+ "consentDeadlineTime": "A String", # The deadline by which the user must provide consent. If consent is not provided by this time, the subscription will be canceled.
+ "newPrice": { # Represents an amount of money with its currency type. # The new price which requires user consent.
+ "currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
+ "nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
+ "units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
+ },
+ "state": "A String", # Output only. The state of the price step-up consent.
+ },
"recurringPrice": { # Represents an amount of money with its currency type. # The current recurring price of the auto renewing plan. Note that the price does not take into account discounts and does not include taxes for tax-exclusive pricing, please call orders.get API instead if transaction details are needed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
diff --git a/docs/dyn/apigee_v1.organizations.developers.balance.html b/docs/dyn/apigee_v1.organizations.developers.balance.html
index f65beff1084..5c2f86fe84c 100644
--- a/docs/dyn/apigee_v1.organizations.developers.balance.html
+++ b/docs/dyn/apigee_v1.organizations.developers.balance.html
@@ -94,7 +94,7 @@ Method Details
The object takes the form of:
{ # Request for AdjustDeveloperBalance.
- "adjustment": { # Represents an amount of money with its currency type. # * A positive value of `adjustment` means that that the API provider wants to adjust the balance for an under-charged developer i.e. the balance of the developer will decrease. * A negative value of `adjustment` means that that the API provider wants to adjust the balance for an over-charged developer i.e. the balance of the developer will increase.
+ "adjustment": { # Represents an amount of money with its currency type. # * A positive value of `adjustment` means that that the API provider wants to adjust the balance for an under-charged developer i.e. the balance of the developer will decrease. * A negative value of `adjustment` means that that the API provider wants to adjust the balance for an over-charged developer i.e. the balance of the developer will increase. NOTE: An adjustment cannot increase the balance of the developer beyond the balance as of the most recent credit. For example, if a developer's balance is updated to be $100, and they spend $10, a negative adjustment can only increase the balance of the developer to $100.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
diff --git a/docs/dyn/apigee_v1.organizations.html b/docs/dyn/apigee_v1.organizations.html
index 0ff4e7cb0ca..f899c165f44 100644
--- a/docs/dyn/apigee_v1.organizations.html
+++ b/docs/dyn/apigee_v1.organizations.html
@@ -179,6 +179,11 @@ Instance Methods
Returns the securityAssessmentResults Resource.
+
+Returns the securityFeedback Resource.
+
securityMonitoringConditions()
diff --git a/docs/dyn/apigee_v1.organizations.securityAssessmentResults.html b/docs/dyn/apigee_v1.organizations.securityAssessmentResults.html
index 72d86c4e856..db24120fe7a 100644
--- a/docs/dyn/apigee_v1.organizations.securityAssessmentResults.html
+++ b/docs/dyn/apigee_v1.organizations.securityAssessmentResults.html
@@ -107,7 +107,7 @@ Method Details
"pageSize": 42, # Optional. The maximum number of results to return. The service may return fewer than this value. If unspecified, at most 50 results will be returned.
"pageToken": "A String", # Optional. A page token, received from a previous `BatchComputeSecurityAssessmentResults` call. Provide this to retrieve the subsequent page.
"profile": "A String", # Required. Name of the profile that is used for computation.
- "scope": "A String", # Required. Scope of the resources for the computation. For Apigee, the environment is the scope of the resources.
+ "scope": "A String", # Optional. Scope of the resources for the computation. For Apigee, the environment is the scope of the resources.
}
x__xgafv: string, V1 error format.
diff --git a/docs/dyn/apigee_v1.organizations.securityFeedback.html b/docs/dyn/apigee_v1.organizations.securityFeedback.html
new file mode 100644
index 00000000000..3014b0381e2
--- /dev/null
+++ b/docs/dyn/apigee_v1.organizations.securityFeedback.html
@@ -0,0 +1,319 @@
+
+
+
+Apigee API . organizations . securityFeedback
+Instance Methods
+
+ close()
+Close httplib2 connections.
+
+ create(parent, body=None, securityFeedbackId=None, x__xgafv=None)
+Creates a new report containing customer feedback.
+
+Deletes a specific feedback report. Used for "undo" of a feedback submission.
+
+Gets a specific customer feedback report.
+
+ list(parent, pageSize=None, pageToken=None, x__xgafv=None)
+Lists all feedback reports which have already been submitted.
+
+Retrieves the next page of results.
+
+ patch(name, body=None, updateMask=None, x__xgafv=None)
+Updates a specific feedback report.
+Method Details
+
+ close()
+ Close httplib2 connections.
+
+
+
+ create(parent, body=None, securityFeedbackId=None, x__xgafv=None)
+ Creates a new report containing customer feedback.
+
+Args:
+ parent: string, Required. Name of the organization. Use the following structure in your request: `organizations/{org}`. (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # Represents a feedback report from an Advanced API Security customer.
+ "comment": "A String", # Optional. Optional text the user can provide for additional, unstructured context.
+ "createTime": "A String", # Output only. The time when this specific feedback id was created.
+ "displayName": "A String", # Optional. The display name of the feedback.
+ "feedbackContexts": [ # Required. One or more attribute/value pairs for constraining the feedback.
+ { # FeedbackContext captures the intent of the submitted feedback.
+ "attribute": "A String", # Required. The attribute the user is providing feedback about.
+ "values": [ # Required. The values of the attribute the user is providing feedback about.
+ "A String",
+ ],
+ },
+ ],
+ "feedbackType": "A String", # Required. The type of feedback being submitted.
+ "name": "A String", # Output only. Identifier. The feedback name is intended to be a system-generated uuid.
+ "reason": "A String", # Optional. The reason for the feedback.
+ "updateTime": "A String", # Output only. The time when this specific feedback id was updated.
+}
+
+ securityFeedbackId: string, Optional. The id for this feedback report. If not provided, it will be set to a system-generated UUID.
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # Represents a feedback report from an Advanced API Security customer.
+ "comment": "A String", # Optional. Optional text the user can provide for additional, unstructured context.
+ "createTime": "A String", # Output only. The time when this specific feedback id was created.
+ "displayName": "A String", # Optional. The display name of the feedback.
+ "feedbackContexts": [ # Required. One or more attribute/value pairs for constraining the feedback.
+ { # FeedbackContext captures the intent of the submitted feedback.
+ "attribute": "A String", # Required. The attribute the user is providing feedback about.
+ "values": [ # Required. The values of the attribute the user is providing feedback about.
+ "A String",
+ ],
+ },
+ ],
+ "feedbackType": "A String", # Required. The type of feedback being submitted.
+ "name": "A String", # Output only. Identifier. The feedback name is intended to be a system-generated uuid.
+ "reason": "A String", # Optional. The reason for the feedback.
+ "updateTime": "A String", # Output only. The time when this specific feedback id was updated.
+}
+
+
+
+ delete(name, x__xgafv=None)
+ Deletes a specific feedback report. Used for "undo" of a feedback submission.
+
+Args:
+ name: string, Required. Name of the SecurityFeedback to delete. Use the following structure in your request: `organizations/{org}/securityFeedback/{feedback_id}` (required)
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); }
+}
+
+
+
+ get(name, x__xgafv=None)
+ Gets a specific customer feedback report.
+
+Args:
+ name: string, Required. Name of the SecurityFeedback. Format: `organizations/{org}/securityFeedback/{feedback_id}` Example: organizations/apigee-organization-name/securityFeedback/feedback-id (required)
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # Represents a feedback report from an Advanced API Security customer.
+ "comment": "A String", # Optional. Optional text the user can provide for additional, unstructured context.
+ "createTime": "A String", # Output only. The time when this specific feedback id was created.
+ "displayName": "A String", # Optional. The display name of the feedback.
+ "feedbackContexts": [ # Required. One or more attribute/value pairs for constraining the feedback.
+ { # FeedbackContext captures the intent of the submitted feedback.
+ "attribute": "A String", # Required. The attribute the user is providing feedback about.
+ "values": [ # Required. The values of the attribute the user is providing feedback about.
+ "A String",
+ ],
+ },
+ ],
+ "feedbackType": "A String", # Required. The type of feedback being submitted.
+ "name": "A String", # Output only. Identifier. The feedback name is intended to be a system-generated uuid.
+ "reason": "A String", # Optional. The reason for the feedback.
+ "updateTime": "A String", # Output only. The time when this specific feedback id was updated.
+}
+
+
+
+ list(parent, pageSize=None, pageToken=None, x__xgafv=None)
+ Lists all feedback reports which have already been submitted.
+
+Args:
+ parent: string, Required. Name of the organization. Format: `organizations/{org}`. Example: organizations/apigee-organization-name/securityFeedback (required)
+ pageSize: integer, Optional. The maximum number of feedback reports to return. The service may return fewer than this value. LINT.IfChange(documented_page_size_limits) If unspecified, at most 10 feedback reports will be returned. The maximum value is 100; values above 100 will be coerced to 100. LINT.ThenChange( //depot/google3/edge/sense/boq/service/v1/securityfeedback/securityfeedback_rpc.go:page_size_limits )
+ pageToken: string, Optional. A page token, received from a previous `ListSecurityFeedback` call. Provide this to retrieve the subsequent page. When paginating, all other parameters provided to `ListSecurityFeedback` must match the call that provided the page token.
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # Response for ListSecurityFeedback
+ "nextPageToken": "A String", # A token that can be sent as `page_token` in `ListSecurityFeedbackRequest` to retrieve the next page. If this field is omitted, there are no subsequent pages.
+ "securityFeedback": [ # List of SecurityFeedback reports.
+ { # Represents a feedback report from an Advanced API Security customer.
+ "comment": "A String", # Optional. Optional text the user can provide for additional, unstructured context.
+ "createTime": "A String", # Output only. The time when this specific feedback id was created.
+ "displayName": "A String", # Optional. The display name of the feedback.
+ "feedbackContexts": [ # Required. One or more attribute/value pairs for constraining the feedback.
+ { # FeedbackContext captures the intent of the submitted feedback.
+ "attribute": "A String", # Required. The attribute the user is providing feedback about.
+ "values": [ # Required. The values of the attribute the user is providing feedback about.
+ "A String",
+ ],
+ },
+ ],
+ "feedbackType": "A String", # Required. The type of feedback being submitted.
+ "name": "A String", # Output only. Identifier. The feedback name is intended to be a system-generated uuid.
+ "reason": "A String", # Optional. The reason for the feedback.
+ "updateTime": "A String", # Output only. The time when this specific feedback id was updated.
+ },
+ ],
+}
+
+
+
+ list_next()
+ Retrieves the next page of results.
+
+ Args:
+ previous_request: The request for the previous page. (required)
+ previous_response: The response from the request for the previous page. (required)
+
+ Returns:
+ A request object that you can call 'execute()' on to request the next
+ page. Returns None if there are no more items in the collection.
+
+
+
+
+ patch(name, body=None, updateMask=None, x__xgafv=None)
+ Updates a specific feedback report.
+
+Args:
+ name: string, Output only. Identifier. The feedback name is intended to be a system-generated uuid. (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # Represents a feedback report from an Advanced API Security customer.
+ "comment": "A String", # Optional. Optional text the user can provide for additional, unstructured context.
+ "createTime": "A String", # Output only. The time when this specific feedback id was created.
+ "displayName": "A String", # Optional. The display name of the feedback.
+ "feedbackContexts": [ # Required. One or more attribute/value pairs for constraining the feedback.
+ { # FeedbackContext captures the intent of the submitted feedback.
+ "attribute": "A String", # Required. The attribute the user is providing feedback about.
+ "values": [ # Required. The values of the attribute the user is providing feedback about.
+ "A String",
+ ],
+ },
+ ],
+ "feedbackType": "A String", # Required. The type of feedback being submitted.
+ "name": "A String", # Output only. Identifier. The feedback name is intended to be a system-generated uuid.
+ "reason": "A String", # Optional. The reason for the feedback.
+ "updateTime": "A String", # Output only. The time when this specific feedback id was updated.
+}
+
+ updateMask: string, Optional. The list of fields to update.
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # Represents a feedback report from an Advanced API Security customer.
+ "comment": "A String", # Optional. Optional text the user can provide for additional, unstructured context.
+ "createTime": "A String", # Output only. The time when this specific feedback id was created.
+ "displayName": "A String", # Optional. The display name of the feedback.
+ "feedbackContexts": [ # Required. One or more attribute/value pairs for constraining the feedback.
+ { # FeedbackContext captures the intent of the submitted feedback.
+ "attribute": "A String", # Required. The attribute the user is providing feedback about.
+ "values": [ # Required. The values of the attribute the user is providing feedback about.
+ "A String",
+ ],
+ },
+ ],
+ "feedbackType": "A String", # Required. The type of feedback being submitted.
+ "name": "A String", # Output only. Identifier. The feedback name is intended to be a system-generated uuid.
+ "reason": "A String", # Optional. The reason for the feedback.
+ "updateTime": "A String", # Output only. The time when this specific feedback id was updated.
+}
+
+
+
\ No newline at end of file
diff --git a/docs/dyn/apigee_v1.organizations.securityMonitoringConditions.html b/docs/dyn/apigee_v1.organizations.securityMonitoringConditions.html
index 8382b6c9e64..fa6bb044ebf 100644
--- a/docs/dyn/apigee_v1.organizations.securityMonitoringConditions.html
+++ b/docs/dyn/apigee_v1.organizations.securityMonitoringConditions.html
@@ -124,7 +124,7 @@ Method Details
},
"name": "A String", # Identifier. Name of the security monitoring condition resource. Format: organizations/{org}/securityMonitoringConditions/{security_monitoring_condition}
"profile": "A String", # Required. ID of security profile of the security monitoring condition.
- "scope": "A String", # Required. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
+ "scope": "A String", # Optional. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
"totalDeployedResources": 42, # Output only. Total number of deployed resources within scope.
"totalMonitoredResources": 42, # Output only. Total number of monitored resources within this condition.
"updateTime": "A String", # Output only. The time of the security monitoring condition update.
@@ -153,7 +153,7 @@ Method Details
},
"name": "A String", # Identifier. Name of the security monitoring condition resource. Format: organizations/{org}/securityMonitoringConditions/{security_monitoring_condition}
"profile": "A String", # Required. ID of security profile of the security monitoring condition.
- "scope": "A String", # Required. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
+ "scope": "A String", # Optional. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
"totalDeployedResources": 42, # Output only. Total number of deployed resources within scope.
"totalMonitoredResources": 42, # Output only. Total number of monitored resources within this condition.
"updateTime": "A String", # Output only. The time of the security monitoring condition update.
@@ -206,7 +206,7 @@ Method Details
},
"name": "A String", # Identifier. Name of the security monitoring condition resource. Format: organizations/{org}/securityMonitoringConditions/{security_monitoring_condition}
"profile": "A String", # Required. ID of security profile of the security monitoring condition.
- "scope": "A String", # Required. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
+ "scope": "A String", # Optional. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
"totalDeployedResources": 42, # Output only. Total number of deployed resources within scope.
"totalMonitoredResources": 42, # Output only. Total number of monitored resources within this condition.
"updateTime": "A String", # Output only. The time of the security monitoring condition update.
@@ -247,7 +247,7 @@ Method Details
},
"name": "A String", # Identifier. Name of the security monitoring condition resource. Format: organizations/{org}/securityMonitoringConditions/{security_monitoring_condition}
"profile": "A String", # Required. ID of security profile of the security monitoring condition.
- "scope": "A String", # Required. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
+ "scope": "A String", # Optional. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
"totalDeployedResources": 42, # Output only. Total number of deployed resources within scope.
"totalMonitoredResources": 42, # Output only. Total number of monitored resources within this condition.
"updateTime": "A String", # Output only. The time of the security monitoring condition update.
@@ -293,7 +293,7 @@ Method Details
},
"name": "A String", # Identifier. Name of the security monitoring condition resource. Format: organizations/{org}/securityMonitoringConditions/{security_monitoring_condition}
"profile": "A String", # Required. ID of security profile of the security monitoring condition.
- "scope": "A String", # Required. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
+ "scope": "A String", # Optional. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
"totalDeployedResources": 42, # Output only. Total number of deployed resources within scope.
"totalMonitoredResources": 42, # Output only. Total number of monitored resources within this condition.
"updateTime": "A String", # Output only. The time of the security monitoring condition update.
@@ -322,7 +322,7 @@ Method Details
},
"name": "A String", # Identifier. Name of the security monitoring condition resource. Format: organizations/{org}/securityMonitoringConditions/{security_monitoring_condition}
"profile": "A String", # Required. ID of security profile of the security monitoring condition.
- "scope": "A String", # Required. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
+ "scope": "A String", # Optional. Scope of the security monitoring condition. For Apigee, the environment is the scope of the resources.
"totalDeployedResources": 42, # Output only. Total number of deployed resources within scope.
"totalMonitoredResources": 42, # Output only. Total number of monitored resources within this condition.
"updateTime": "A String", # Output only. The time of the security monitoring condition update.
diff --git a/docs/dyn/apihub_v1.projects.locations.apiHubInstances.html b/docs/dyn/apihub_v1.projects.locations.apiHubInstances.html
index 918a0b2894d..f4a57a5592f 100644
--- a/docs/dyn/apihub_v1.projects.locations.apiHubInstances.html
+++ b/docs/dyn/apihub_v1.projects.locations.apiHubInstances.html
@@ -82,7 +82,7 @@ Instance Methods
Provisions instance resources for the API Hub.
-Deletes the API hub instance.
+Deletes the API hub instance. Deleting the API hub instance will also result in the removal of all associated runtime project attachments and the host project registration.
Gets details of a single API Hub instance.
@@ -154,7 +154,7 @@ Method Details
delete(name, x__xgafv=None)
- Deletes the API hub instance.
+ Deletes the API hub instance. Deleting the API hub instance will also result in the removal of all associated runtime project attachments and the host project registration.
Args:
name: string, Required. The name of the Api Hub instance to delete. Format: `projects/{project}/locations/{location}/apiHubInstances/{apiHubInstance}`. (required)
diff --git a/docs/dyn/apihub_v1.projects.locations.html b/docs/dyn/apihub_v1.projects.locations.html
index 618ec6585e1..bee6b7afaaf 100644
--- a/docs/dyn/apihub_v1.projects.locations.html
+++ b/docs/dyn/apihub_v1.projects.locations.html
@@ -1036,7 +1036,7 @@ Method Details
Args:
name: string, The resource that owns the locations collection, if applicable. (required)
- extraLocationTypes: string, Optional. Do not use this field. It is unsupported and is ignored unless explicitly documented otherwise. This is primarily for internal usage. (repeated)
+ extraLocationTypes: string, Optional. Unless explicitly documented otherwise, don't use this unsupported field which is primarily intended for internal usage. (repeated)
filter: string, A filter to narrow down results to a preferred subset. The filtering language accepts strings like `"displayName=tokyo"`, and is documented in more detail in [AIP-160](https://google.aip.dev/160).
pageSize: integer, The maximum number of results to return. If not set, the service selects a default.
pageToken: string, A page token received from the `next_page_token` field in the response. Send that page token to receive the subsequent page.
diff --git a/docs/dyn/apihub_v1.projects.locations.plugins.html b/docs/dyn/apihub_v1.projects.locations.plugins.html
index f9f1e573d6f..7b474421740 100644
--- a/docs/dyn/apihub_v1.projects.locations.plugins.html
+++ b/docs/dyn/apihub_v1.projects.locations.plugins.html
@@ -130,7 +130,7 @@ Method Details
The object takes the form of:
{ # A plugin resource in the API Hub.
- "actionsConfig": [ # Required. The configuration of actions supported by the plugin.
+ "actionsConfig": [ # Optional. The configuration of actions supported by the plugin. **REQUIRED**: This field must be provided when creating or updating a Plugin. The server will reject requests if this field is missing.
{ # PluginActionConfig represents the configuration of an action supported by a plugin.
"description": "A String", # Required. The description of the operation performed by the action.
"displayName": "A String", # Required. The display name of the action.
@@ -226,7 +226,7 @@ Method Details
An object of the form:
{ # A plugin resource in the API Hub.
- "actionsConfig": [ # Required. The configuration of actions supported by the plugin.
+ "actionsConfig": [ # Optional. The configuration of actions supported by the plugin. **REQUIRED**: This field must be provided when creating or updating a Plugin. The server will reject requests if this field is missing.
{ # PluginActionConfig represents the configuration of an action supported by a plugin.
"description": "A String", # Required. The description of the operation performed by the action.
"displayName": "A String", # Required. The display name of the action.
@@ -369,7 +369,7 @@ Method Details
An object of the form:
{ # A plugin resource in the API Hub.
- "actionsConfig": [ # Required. The configuration of actions supported by the plugin.
+ "actionsConfig": [ # Optional. The configuration of actions supported by the plugin. **REQUIRED**: This field must be provided when creating or updating a Plugin. The server will reject requests if this field is missing.
{ # PluginActionConfig represents the configuration of an action supported by a plugin.
"description": "A String", # Required. The description of the operation performed by the action.
"displayName": "A String", # Required. The display name of the action.
@@ -477,7 +477,7 @@ Method Details
An object of the form:
{ # A plugin resource in the API Hub.
- "actionsConfig": [ # Required. The configuration of actions supported by the plugin.
+ "actionsConfig": [ # Optional. The configuration of actions supported by the plugin. **REQUIRED**: This field must be provided when creating or updating a Plugin. The server will reject requests if this field is missing.
{ # PluginActionConfig represents the configuration of an action supported by a plugin.
"description": "A String", # Required. The description of the operation performed by the action.
"displayName": "A String", # Required. The display name of the action.
@@ -579,7 +579,7 @@ Method Details
An object of the form:
{ # A plugin resource in the API Hub.
- "actionsConfig": [ # Required. The configuration of actions supported by the plugin.
+ "actionsConfig": [ # Optional. The configuration of actions supported by the plugin. **REQUIRED**: This field must be provided when creating or updating a Plugin. The server will reject requests if this field is missing.
{ # PluginActionConfig represents the configuration of an action supported by a plugin.
"description": "A String", # Required. The description of the operation performed by the action.
"displayName": "A String", # Required. The display name of the action.
@@ -711,7 +711,7 @@ Method Details
"nextPageToken": "A String", # A token, which can be sent as `page_token` to retrieve the next page. If this field is omitted, there are no subsequent pages.
"plugins": [ # The plugins from the specified parent resource.
{ # A plugin resource in the API Hub.
- "actionsConfig": [ # Required. The configuration of actions supported by the plugin.
+ "actionsConfig": [ # Optional. The configuration of actions supported by the plugin. **REQUIRED**: This field must be provided when creating or updating a Plugin. The server will reject requests if this field is missing.
{ # PluginActionConfig represents the configuration of an action supported by a plugin.
"description": "A String", # Required. The description of the operation performed by the action.
"displayName": "A String", # Required. The display name of the action.
diff --git a/docs/dyn/apihub_v1.projects.locations.plugins.instances.html b/docs/dyn/apihub_v1.projects.locations.plugins.instances.html
index bb9f08c8cc7..221d4305e00 100644
--- a/docs/dyn/apihub_v1.projects.locations.plugins.instances.html
+++ b/docs/dyn/apihub_v1.projects.locations.plugins.instances.html
@@ -101,6 +101,9 @@ Instance Methods
Retrieves the next page of results.
+
+ manageSourceData(name, body=None, x__xgafv=None)
+Manages data for a given plugin instance.
patch(name, body=None, updateMask=None, x__xgafv=None)
Updates a plugin instance in the API hub. The following fields in the plugin_instance can be updated currently: * display_name * schedule_cron_expression The update_mask should be used to specify the fields being updated. To update the auth_config and additional_config of the plugin instance, use the ApplyPluginInstanceConfig method.
@@ -658,6 +661,34 @@ Method Details
+
+ manageSourceData(name, body=None, x__xgafv=None)
+ Manages data for a given plugin instance.
+
+Args:
+ name: string, Required. The name of the plugin instance for which data needs to be managed. Format: `projects/{project}/locations/{location}/plugins/{plugin}/instances/{instance}` (required)
+ body: object, The request body.
+ The object takes the form of:
+
+{ # The ManagePluginInstanceSourceData method's request.
+ "action": "A String", # Required. Action to be performed.
+ "data": "A String", # Required. Data to be managed.
+ "dataType": "A String", # Required. Type of data to be managed.
+ "relativePath": "A String", # Required. Relative path of data being managed for a given plugin instance.
+}
+
+ x__xgafv: string, V1 error format.
+ Allowed values
+ 1 - v1 error format
+ 2 - v2 error format
+
+Returns:
+ An object of the form:
+
+ { # The ManagePluginInstanceSourceData method's response.
+}
+
+
patch(name, body=None, updateMask=None, x__xgafv=None)
Updates a plugin instance in the API hub. The following fields in the plugin_instance can be updated currently: * display_name * schedule_cron_expression The update_mask should be used to specify the fields being updated. To update the auth_config and additional_config of the plugin instance, use the ApplyPluginInstanceConfig method.
diff --git a/docs/dyn/authorizedbuyersmarketplace_v1alpha.buyers.dataSegments.html b/docs/dyn/authorizedbuyersmarketplace_v1alpha.buyers.dataSegments.html
index 7aa12545def..19177d57aea 100644
--- a/docs/dyn/authorizedbuyersmarketplace_v1alpha.buyers.dataSegments.html
+++ b/docs/dyn/authorizedbuyersmarketplace_v1alpha.buyers.dataSegments.html
@@ -120,7 +120,7 @@ Method Details
An object of the form:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -147,7 +147,7 @@ Method Details
The object takes the form of:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -167,7 +167,7 @@ Method Details
An object of the form:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -200,7 +200,7 @@ Method Details
An object of the form:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -227,7 +227,7 @@ Method Details
An object of the form:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -258,7 +258,7 @@ Method Details
{ # Response message for listing data segments.
"dataSegments": [ # The list of data segments.
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -297,7 +297,7 @@ Method Details
The object takes the form of:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -318,7 +318,7 @@ Method Details
An object of the form:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
diff --git a/docs/dyn/authorizedbuyersmarketplace_v1beta.curators.dataSegments.html b/docs/dyn/authorizedbuyersmarketplace_v1beta.curators.dataSegments.html
index abdc5f20c74..9c300d9ef9d 100644
--- a/docs/dyn/authorizedbuyersmarketplace_v1beta.curators.dataSegments.html
+++ b/docs/dyn/authorizedbuyersmarketplace_v1beta.curators.dataSegments.html
@@ -120,7 +120,7 @@ Method Details
An object of the form:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -147,7 +147,7 @@ Method Details
The object takes the form of:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -167,7 +167,7 @@ Method Details
An object of the form:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -200,7 +200,7 @@ Method Details
An object of the form:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -227,7 +227,7 @@ Method Details
An object of the form:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -258,7 +258,7 @@ Method Details
{ # Response message for listing data segments.
"dataSegments": [ # The list of data segments.
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -297,7 +297,7 @@ Method Details
The object takes the form of:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
@@ -318,7 +318,7 @@ Method Details
An object of the form:
{ # Defines an identifier for a segment of inventory that can be targeted by curators or media planners in the deals or auction packages UI. Curation of inventory is done by curators on external platforms.
- "cpmFee": { # Represents an amount of money with its currency type. # Required. This will be charged when other accounts use this data segment. For example, when other accounts add this data segment to a deal or auction package. Once set, the currency code cannot be changed.
+ "cpmFee": { # Represents an amount of money with its currency type. # Optional. A fixed fee charged per thousand impressions. Once set, the currency code cannot be changed.
"currencyCode": "A String", # The three-letter currency code defined in ISO 4217.
"nanos": 42, # Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
"units": "A String", # The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
diff --git a/docs/dyn/backupdr_v1.projects.locations.backupPlans.html b/docs/dyn/backupdr_v1.projects.locations.backupPlans.html
index 19196f15cc7..455cf896e3a 100644
--- a/docs/dyn/backupdr_v1.projects.locations.backupPlans.html
+++ b/docs/dyn/backupdr_v1.projects.locations.backupPlans.html
@@ -118,7 +118,7 @@ Method Details
{ # A `BackupPlan` specifies some common fields, such as `description` as well as one or more `BackupRule` messages. Each `BackupRule` has a retention policy and defines a schedule by which the system is to perform backup workloads.
"backupRules": [ # Optional. The backup rules for this `BackupPlan`.
{ # `BackupRule` binds the backup schedule to a retention policy.
- "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules.
+ "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules. Note: Longer retention can lead to higher storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"ruleId": "A String", # Required. Immutable. The unique id of this `BackupRule`. The `rule_id` is unique per `BackupPlan`.The `rule_id` must start with a lowercase letter followed by up to 62 lowercase letters, numbers, or hyphens. Pattern, /a-z{,62}/.
"standardSchedule": { # `StandardSchedule` defines a schedule that run within the confines of a defined window of days. We can define recurrence type for schedule as HOURLY, DAILY, WEEKLY, MONTHLY or YEARLY. # Optional. Defines a schedule that runs within the confines of a defined window of time.
"backupWindow": { # `BackupWindow` defines a window of the day during which backup jobs will run. # Required. A BackupWindow defines the window of day during which backup jobs will run. Jobs are queued at the beginning of the window and will be marked as `NOT_RUN` if they do not start by the end of the window. Note: running jobs will not be cancelled at the end of the window.
@@ -248,7 +248,7 @@ Method Details
{ # A `BackupPlan` specifies some common fields, such as `description` as well as one or more `BackupRule` messages. Each `BackupRule` has a retention policy and defines a schedule by which the system is to perform backup workloads.
"backupRules": [ # Optional. The backup rules for this `BackupPlan`.
{ # `BackupRule` binds the backup schedule to a retention policy.
- "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules.
+ "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules. Note: Longer retention can lead to higher storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"ruleId": "A String", # Required. Immutable. The unique id of this `BackupRule`. The `rule_id` is unique per `BackupPlan`.The `rule_id` must start with a lowercase letter followed by up to 62 lowercase letters, numbers, or hyphens. Pattern, /a-z{,62}/.
"standardSchedule": { # `StandardSchedule` defines a schedule that run within the confines of a defined window of days. We can define recurrence type for schedule as HOURLY, DAILY, WEEKLY, MONTHLY or YEARLY. # Optional. Defines a schedule that runs within the confines of a defined window of time.
"backupWindow": { # `BackupWindow` defines a window of the day during which backup jobs will run. # Required. A BackupWindow defines the window of day during which backup jobs will run. Jobs are queued at the beginning of the window and will be marked as `NOT_RUN` if they do not start by the end of the window. Note: running jobs will not be cancelled at the end of the window.
@@ -318,7 +318,7 @@ Method Details
{ # A `BackupPlan` specifies some common fields, such as `description` as well as one or more `BackupRule` messages. Each `BackupRule` has a retention policy and defines a schedule by which the system is to perform backup workloads.
"backupRules": [ # Optional. The backup rules for this `BackupPlan`.
{ # `BackupRule` binds the backup schedule to a retention policy.
- "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules.
+ "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules. Note: Longer retention can lead to higher storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"ruleId": "A String", # Required. Immutable. The unique id of this `BackupRule`. The `rule_id` is unique per `BackupPlan`.The `rule_id` must start with a lowercase letter followed by up to 62 lowercase letters, numbers, or hyphens. Pattern, /a-z{,62}/.
"standardSchedule": { # `StandardSchedule` defines a schedule that run within the confines of a defined window of days. We can define recurrence type for schedule as HOURLY, DAILY, WEEKLY, MONTHLY or YEARLY. # Optional. Defines a schedule that runs within the confines of a defined window of time.
"backupWindow": { # `BackupWindow` defines a window of the day during which backup jobs will run. # Required. A BackupWindow defines the window of day during which backup jobs will run. Jobs are queued at the beginning of the window and will be marked as `NOT_RUN` if they do not start by the end of the window. Note: running jobs will not be cancelled at the end of the window.
@@ -397,7 +397,7 @@ Method Details
{ # A `BackupPlan` specifies some common fields, such as `description` as well as one or more `BackupRule` messages. Each `BackupRule` has a retention policy and defines a schedule by which the system is to perform backup workloads.
"backupRules": [ # Optional. The backup rules for this `BackupPlan`.
{ # `BackupRule` binds the backup schedule to a retention policy.
- "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules.
+ "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules. Note: Longer retention can lead to higher storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"ruleId": "A String", # Required. Immutable. The unique id of this `BackupRule`. The `rule_id` is unique per `BackupPlan`.The `rule_id` must start with a lowercase letter followed by up to 62 lowercase letters, numbers, or hyphens. Pattern, /a-z{,62}/.
"standardSchedule": { # `StandardSchedule` defines a schedule that run within the confines of a defined window of days. We can define recurrence type for schedule as HOURLY, DAILY, WEEKLY, MONTHLY or YEARLY. # Optional. Defines a schedule that runs within the confines of a defined window of time.
"backupWindow": { # `BackupWindow` defines a window of the day during which backup jobs will run. # Required. A BackupWindow defines the window of day during which backup jobs will run. Jobs are queued at the beginning of the window and will be marked as `NOT_RUN` if they do not start by the end of the window. Note: running jobs will not be cancelled at the end of the window.
diff --git a/docs/dyn/backupdr_v1.projects.locations.backupPlans.revisions.html b/docs/dyn/backupdr_v1.projects.locations.backupPlans.revisions.html
index 6fc00d1855d..0e4ce9900ec 100644
--- a/docs/dyn/backupdr_v1.projects.locations.backupPlans.revisions.html
+++ b/docs/dyn/backupdr_v1.projects.locations.backupPlans.revisions.html
@@ -110,7 +110,7 @@ Method Details
"backupPlanSnapshot": { # A `BackupPlan` specifies some common fields, such as `description` as well as one or more `BackupRule` messages. Each `BackupRule` has a retention policy and defines a schedule by which the system is to perform backup workloads. # The Backup Plan being encompassed by this revision.
"backupRules": [ # Optional. The backup rules for this `BackupPlan`.
{ # `BackupRule` binds the backup schedule to a retention policy.
- "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules.
+ "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules. Note: Longer retention can lead to higher storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"ruleId": "A String", # Required. Immutable. The unique id of this `BackupRule`. The `rule_id` is unique per `BackupPlan`.The `rule_id` must start with a lowercase letter followed by up to 62 lowercase letters, numbers, or hyphens. Pattern, /a-z{,62}/.
"standardSchedule": { # `StandardSchedule` defines a schedule that run within the confines of a defined window of days. We can define recurrence type for schedule as HOURLY, DAILY, WEEKLY, MONTHLY or YEARLY. # Optional. Defines a schedule that runs within the confines of a defined window of time.
"backupWindow": { # `BackupWindow` defines a window of the day during which backup jobs will run. # Required. A BackupWindow defines the window of day during which backup jobs will run. Jobs are queued at the beginning of the window and will be marked as `NOT_RUN` if they do not start by the end of the window. Note: running jobs will not be cancelled at the end of the window.
@@ -184,7 +184,7 @@ Method Details
"backupPlanSnapshot": { # A `BackupPlan` specifies some common fields, such as `description` as well as one or more `BackupRule` messages. Each `BackupRule` has a retention policy and defines a schedule by which the system is to perform backup workloads. # The Backup Plan being encompassed by this revision.
"backupRules": [ # Optional. The backup rules for this `BackupPlan`.
{ # `BackupRule` binds the backup schedule to a retention policy.
- "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules.
+ "backupRetentionDays": 42, # Required. Configures the duration for which backup data will be kept. It is defined in “days”. The value should be greater than or equal to minimum enforced retention of the backup vault. Minimum value is 1 and maximum value is 36159 for custom retention on-demand backup. Minimum and maximum values are workload specific for all other rules. Note: Longer retention can lead to higher storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"ruleId": "A String", # Required. Immutable. The unique id of this `BackupRule`. The `rule_id` is unique per `BackupPlan`.The `rule_id` must start with a lowercase letter followed by up to 62 lowercase letters, numbers, or hyphens. Pattern, /a-z{,62}/.
"standardSchedule": { # `StandardSchedule` defines a schedule that run within the confines of a defined window of days. We can define recurrence type for schedule as HOURLY, DAILY, WEEKLY, MONTHLY or YEARLY. # Optional. Defines a schedule that runs within the confines of a defined window of time.
"backupWindow": { # `BackupWindow` defines a window of the day during which backup jobs will run. # Required. A BackupWindow defines the window of day during which backup jobs will run. Jobs are queued at the beginning of the window and will be marked as `NOT_RUN` if they do not start by the end of the window. Note: running jobs will not be cancelled at the end of the window.
diff --git a/docs/dyn/backupdr_v1.projects.locations.backupVaults.html b/docs/dyn/backupdr_v1.projects.locations.backupVaults.html
index 65f784384c1..8f314278f9f 100644
--- a/docs/dyn/backupdr_v1.projects.locations.backupVaults.html
+++ b/docs/dyn/backupdr_v1.projects.locations.backupVaults.html
@@ -130,7 +130,7 @@ Method Details
"a_key": "A String",
},
"backupCount": "A String", # Output only. The number of backups in this backup vault.
- "backupMinimumEnforcedRetentionDuration": "A String", # Required. The default and minimum enforced retention for each backup within the backup vault. The enforced retention for each backup can be extended.
+ "backupMinimumEnforcedRetentionDuration": "A String", # Required. The default and minimum enforced retention for each backup within the backup vault. The enforced retention for each backup can be extended. Note: Longer minimum enforced retention period impacts potential storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"createTime": "A String", # Output only. The time when the instance was created.
"deletable": True or False, # Output only. Set to true when there are no backups nested under this resource.
"description": "A String", # Optional. The description of the BackupVault instance (2048 characters or less).
@@ -246,7 +246,7 @@ Method Details
"a_key": "A String",
},
"backupCount": "A String", # Output only. The number of backups in this backup vault.
- "backupMinimumEnforcedRetentionDuration": "A String", # Required. The default and minimum enforced retention for each backup within the backup vault. The enforced retention for each backup can be extended.
+ "backupMinimumEnforcedRetentionDuration": "A String", # Required. The default and minimum enforced retention for each backup within the backup vault. The enforced retention for each backup can be extended. Note: Longer minimum enforced retention period impacts potential storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"createTime": "A String", # Output only. The time when the instance was created.
"deletable": True or False, # Output only. Set to true when there are no backups nested under this resource.
"description": "A String", # Optional. The description of the BackupVault instance (2048 characters or less).
@@ -309,7 +309,7 @@ Method Details
"a_key": "A String",
},
"backupCount": "A String", # Output only. The number of backups in this backup vault.
- "backupMinimumEnforcedRetentionDuration": "A String", # Required. The default and minimum enforced retention for each backup within the backup vault. The enforced retention for each backup can be extended.
+ "backupMinimumEnforcedRetentionDuration": "A String", # Required. The default and minimum enforced retention for each backup within the backup vault. The enforced retention for each backup can be extended. Note: Longer minimum enforced retention period impacts potential storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"createTime": "A String", # Output only. The time when the instance was created.
"deletable": True or False, # Output only. Set to true when there are no backups nested under this resource.
"description": "A String", # Optional. The description of the BackupVault instance (2048 characters or less).
@@ -358,7 +358,7 @@ Method Details
"a_key": "A String",
},
"backupCount": "A String", # Output only. The number of backups in this backup vault.
- "backupMinimumEnforcedRetentionDuration": "A String", # Required. The default and minimum enforced retention for each backup within the backup vault. The enforced retention for each backup can be extended.
+ "backupMinimumEnforcedRetentionDuration": "A String", # Required. The default and minimum enforced retention for each backup within the backup vault. The enforced retention for each backup can be extended. Note: Longer minimum enforced retention period impacts potential storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"createTime": "A String", # Output only. The time when the instance was created.
"deletable": True or False, # Output only. Set to true when there are no backups nested under this resource.
"description": "A String", # Optional. The description of the BackupVault instance (2048 characters or less).
@@ -411,7 +411,7 @@ Method Details
"a_key": "A String",
},
"backupCount": "A String", # Output only. The number of backups in this backup vault.
- "backupMinimumEnforcedRetentionDuration": "A String", # Required. The default and minimum enforced retention for each backup within the backup vault. The enforced retention for each backup can be extended.
+ "backupMinimumEnforcedRetentionDuration": "A String", # Required. The default and minimum enforced retention for each backup within the backup vault. The enforced retention for each backup can be extended. Note: Longer minimum enforced retention period impacts potential storage costs post introductory trial. We recommend starting with a short duration of 3 days or less.
"createTime": "A String", # Output only. The time when the instance was created.
"deletable": True or False, # Output only. Set to true when there are no backups nested under this resource.
"description": "A String", # Optional. The description of the BackupVault instance (2048 characters or less).
diff --git a/docs/dyn/backupdr_v1.projects.locations.html b/docs/dyn/backupdr_v1.projects.locations.html
index f45fca047d6..ece1123bbfd 100644
--- a/docs/dyn/backupdr_v1.projects.locations.html
+++ b/docs/dyn/backupdr_v1.projects.locations.html
@@ -182,6 +182,7 @@ Method Details
An object of the form:
{ # Represents a Trial for a project.
+ "endReason": "A String", # Output only. The reason for ending the trial.
"endTime": "A String", # Output only. The time when the trial will expire.
"name": "A String", # Identifier. The resource name of the trial. Format: projects/{project}/locations/{location}/trial
"startTime": "A String", # Output only. The time when the trial was subscribed.
diff --git a/docs/dyn/backupdr_v1.projects.locations.trial.html b/docs/dyn/backupdr_v1.projects.locations.trial.html
index 01c562291b6..caca1e91569 100644
--- a/docs/dyn/backupdr_v1.projects.locations.trial.html
+++ b/docs/dyn/backupdr_v1.projects.locations.trial.html
@@ -107,6 +107,7 @@ Method Details
An object of the form:
{ # Represents a Trial for a project.
+ "endReason": "A String", # Output only. The reason for ending the trial.
"endTime": "A String", # Output only. The time when the trial will expire.
"name": "A String", # Identifier. The resource name of the trial. Format: projects/{project}/locations/{location}/trial
"startTime": "A String", # Output only. The time when the trial was subscribed.
diff --git a/docs/dyn/bigquerydatatransfer_v1.projects.locations.html b/docs/dyn/bigquerydatatransfer_v1.projects.locations.html
index b328ad6694d..da6dc4c755c 100644
--- a/docs/dyn/bigquerydatatransfer_v1.projects.locations.html
+++ b/docs/dyn/bigquerydatatransfer_v1.projects.locations.html
@@ -168,7 +168,7 @@ Method Details
Args:
name: string, The resource that owns the locations collection, if applicable. (required)
- extraLocationTypes: string, Optional. Do not use this field. It is unsupported and is ignored unless explicitly documented otherwise. This is primarily for internal usage. (repeated)
+ extraLocationTypes: string, Optional. Unless explicitly documented otherwise, don't use this unsupported field which is primarily intended for internal usage. (repeated)
filter: string, A filter to narrow down results to a preferred subset. The filtering language accepts strings like `"displayName=tokyo"`, and is documented in more detail in [AIP-160](https://google.aip.dev/160).
pageSize: integer, The maximum number of results to return. If not set, the service selects a default.
pageToken: string, A page token received from the `next_page_token` field in the response. Send that page token to receive the subsequent page.
diff --git a/docs/dyn/bigqueryreservation_v1.projects.locations.reservations.html b/docs/dyn/bigqueryreservation_v1.projects.locations.reservations.html
index bd9b33990aa..38fe9d256a4 100644
--- a/docs/dyn/bigqueryreservation_v1.projects.locations.reservations.html
+++ b/docs/dyn/bigqueryreservation_v1.projects.locations.reservations.html
@@ -241,7 +241,7 @@ Method Details
The object takes the form of:
{ # The request for ReservationService.FailoverReservation.
- "failoverMode": "A String", # Optional. failover mode for the failover operation.
+ "failoverMode": "A String", # Optional. A parameter that determines how writes that are pending replication are handled after a failover is initiated. If not specified, HARD failover mode is used by default.
}
x__xgafv: string, V1 error format.
diff --git a/docs/dyn/bigtableadmin_v2.projects.instances.materializedViews.html b/docs/dyn/bigtableadmin_v2.projects.instances.materializedViews.html
index e700092d789..69d9ecf9439 100644
--- a/docs/dyn/bigtableadmin_v2.projects.instances.materializedViews.html
+++ b/docs/dyn/bigtableadmin_v2.projects.instances.materializedViews.html
@@ -120,10 +120,10 @@ Method Details
The object takes the form of:
{ # A materialized view object that can be referenced in SQL queries.
- "deletionProtection": True or False, # Set to true to make the MaterializedView protected against deletion.
- "etag": "A String", # Optional. The etag for this materialized view. This may be sent on update requests to ensure that the client has an up-to-date value before proceeding. The server returns an ABORTED error on a mismatched etag.
- "name": "A String", # Identifier. The unique name of the materialized view. Format: `projects/{project}/instances/{instance}/materializedViews/{materialized_view}`
- "query": "A String", # Required. Immutable. The materialized view's select query.
+ "deletionProtection": True or False, # Set to true to make the MaterializedView protected against deletion. Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "etag": "A String", # Optional. The etag for this materialized view. This may be sent on update requests to ensure that the client has an up-to-date value before proceeding. The server returns an ABORTED error on a mismatched etag. Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "name": "A String", # Identifier. The unique name of the materialized view. Format: `projects/{project}/instances/{instance}/materializedViews/{materialized_view}` Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "query": "A String", # Required. Immutable. The materialized view's select query. Views: `SCHEMA_VIEW`, `FULL`.
}
materializedViewId: string, Required. The ID to use for the materialized view, which will become the final component of the materialized view's resource name.
@@ -190,10 +190,10 @@ Method Details
An object of the form:
{ # A materialized view object that can be referenced in SQL queries.
- "deletionProtection": True or False, # Set to true to make the MaterializedView protected against deletion.
- "etag": "A String", # Optional. The etag for this materialized view. This may be sent on update requests to ensure that the client has an up-to-date value before proceeding. The server returns an ABORTED error on a mismatched etag.
- "name": "A String", # Identifier. The unique name of the materialized view. Format: `projects/{project}/instances/{instance}/materializedViews/{materialized_view}`
- "query": "A String", # Required. Immutable. The materialized view's select query.
+ "deletionProtection": True or False, # Set to true to make the MaterializedView protected against deletion. Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "etag": "A String", # Optional. The etag for this materialized view. This may be sent on update requests to ensure that the client has an up-to-date value before proceeding. The server returns an ABORTED error on a mismatched etag. Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "name": "A String", # Identifier. The unique name of the materialized view. Format: `projects/{project}/instances/{instance}/materializedViews/{materialized_view}` Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "query": "A String", # Required. Immutable. The materialized view's select query. Views: `SCHEMA_VIEW`, `FULL`.
}
@@ -272,10 +272,10 @@ Method Details
{ # Response message for BigtableInstanceAdmin.ListMaterializedViews.
"materializedViews": [ # The list of requested materialized views.
{ # A materialized view object that can be referenced in SQL queries.
- "deletionProtection": True or False, # Set to true to make the MaterializedView protected against deletion.
- "etag": "A String", # Optional. The etag for this materialized view. This may be sent on update requests to ensure that the client has an up-to-date value before proceeding. The server returns an ABORTED error on a mismatched etag.
- "name": "A String", # Identifier. The unique name of the materialized view. Format: `projects/{project}/instances/{instance}/materializedViews/{materialized_view}`
- "query": "A String", # Required. Immutable. The materialized view's select query.
+ "deletionProtection": True or False, # Set to true to make the MaterializedView protected against deletion. Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "etag": "A String", # Optional. The etag for this materialized view. This may be sent on update requests to ensure that the client has an up-to-date value before proceeding. The server returns an ABORTED error on a mismatched etag. Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "name": "A String", # Identifier. The unique name of the materialized view. Format: `projects/{project}/instances/{instance}/materializedViews/{materialized_view}` Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "query": "A String", # Required. Immutable. The materialized view's select query. Views: `SCHEMA_VIEW`, `FULL`.
},
],
"nextPageToken": "A String", # A token, which can be sent as `page_token` to retrieve the next page. If this field is omitted, there are no subsequent pages.
@@ -301,15 +301,15 @@ Method Details
Updates a materialized view within an instance.
Args:
- name: string, Identifier. The unique name of the materialized view. Format: `projects/{project}/instances/{instance}/materializedViews/{materialized_view}` (required)
+ name: string, Identifier. The unique name of the materialized view. Format: `projects/{project}/instances/{instance}/materializedViews/{materialized_view}` Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`. (required)
body: object, The request body.
The object takes the form of:
{ # A materialized view object that can be referenced in SQL queries.
- "deletionProtection": True or False, # Set to true to make the MaterializedView protected against deletion.
- "etag": "A String", # Optional. The etag for this materialized view. This may be sent on update requests to ensure that the client has an up-to-date value before proceeding. The server returns an ABORTED error on a mismatched etag.
- "name": "A String", # Identifier. The unique name of the materialized view. Format: `projects/{project}/instances/{instance}/materializedViews/{materialized_view}`
- "query": "A String", # Required. Immutable. The materialized view's select query.
+ "deletionProtection": True or False, # Set to true to make the MaterializedView protected against deletion. Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "etag": "A String", # Optional. The etag for this materialized view. This may be sent on update requests to ensure that the client has an up-to-date value before proceeding. The server returns an ABORTED error on a mismatched etag. Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "name": "A String", # Identifier. The unique name of the materialized view. Format: `projects/{project}/instances/{instance}/materializedViews/{materialized_view}` Views: `SCHEMA_VIEW`, `REPLICATION_VIEW`, `FULL`.
+ "query": "A String", # Required. Immutable. The materialized view's select query. Views: `SCHEMA_VIEW`, `FULL`.
}
updateMask: string, Optional. The list of fields to update.
diff --git a/docs/dyn/classroom_v1.courses.announcements.html b/docs/dyn/classroom_v1.courses.announcements.html
index d7a64bc8ead..42c0371b10c 100644
--- a/docs/dyn/classroom_v1.courses.announcements.html
+++ b/docs/dyn/classroom_v1.courses.announcements.html
@@ -102,7 +102,7 @@ Instance Methods
Retrieves the next page of results.
modifyAssignees(courseId, id, body=None, x__xgafv=None)
-Modifies assignee mode and options of an announcement. Only a teacher of the course that contains the announcement may call this method. This method returns the following error codes: * `PERMISSION_DENIED` if the requesting user is not permitted to access the requested course or course work or for access errors. * `INVALID_ARGUMENT` if the request is malformed. * `NOT_FOUND` if the requested course or course work does not exist.
+Modifies assignee mode and options of an announcement. Only a teacher of the course that contains the announcement may call this method. This method returns the following error codes: * `PERMISSION_DENIED` if the requesting user is not permitted to access the requested course or course work or for access errors. * `INVALID_ARGUMENT` if the request is malformed. * `NOT_FOUND` if the requested course or course work does not exist. * `FAILED_PRECONDITION` for the following request error: * EmptyAssignees
patch(courseId, id, body=None, updateMask=None, x__xgafv=None)
Updates one or more fields of an announcement. This method returns the following error codes: * `PERMISSION_DENIED` if the requesting developer project did not create the corresponding announcement or for access errors. * `INVALID_ARGUMENT` if the request is malformed. * `FAILED_PRECONDITION` if the requested announcement has already been deleted. * `NOT_FOUND` if the requested course or announcement does not exist
@@ -473,7 +473,7 @@ Method Details
modifyAssignees(courseId, id, body=None, x__xgafv=None)
- Modifies assignee mode and options of an announcement. Only a teacher of the course that contains the announcement may call this method. This method returns the following error codes: * `PERMISSION_DENIED` if the requesting user is not permitted to access the requested course or course work or for access errors. * `INVALID_ARGUMENT` if the request is malformed. * `NOT_FOUND` if the requested course or course work does not exist.
+ Modifies assignee mode and options of an announcement. Only a teacher of the course that contains the announcement may call this method. This method returns the following error codes: * `PERMISSION_DENIED` if the requesting user is not permitted to access the requested course or course work or for access errors. * `INVALID_ARGUMENT` if the request is malformed. * `NOT_FOUND` if the requested course or course work does not exist. * `FAILED_PRECONDITION` for the following request error: * EmptyAssignees
Args:
courseId: string, Identifier of the course. This identifier can be either the Classroom-assigned identifier or an alias. (required)
diff --git a/docs/dyn/classroom_v1.courses.courseWork.html b/docs/dyn/classroom_v1.courses.courseWork.html
index 47c597c5dc3..9bd10ff2ac9 100644
--- a/docs/dyn/classroom_v1.courses.courseWork.html
+++ b/docs/dyn/classroom_v1.courses.courseWork.html
@@ -112,7 +112,7 @@ Instance Methods
Retrieves the next page of results.
modifyAssignees(courseId, id, body=None, x__xgafv=None)
-Modifies assignee mode and options of a coursework. Only a teacher of the course that contains the coursework may call this method. This method returns the following error codes: * `PERMISSION_DENIED` if the requesting user is not permitted to access the requested course or course work or for access errors. * `INVALID_ARGUMENT` if the request is malformed. * `NOT_FOUND` if the requested course or course work does not exist.
+Modifies assignee mode and options of a coursework. Only a teacher of the course that contains the coursework may call this method. This method returns the following error codes: * `PERMISSION_DENIED` if the requesting user is not permitted to access the requested course or course work or for access errors. * `INVALID_ARGUMENT` if the request is malformed. * `NOT_FOUND` if the requested course or course work does not exist. * `FAILED_PRECONDITION` for the following request error: * EmptyAssignees
patch(courseId, id, body=None, updateMask=None, x__xgafv=None)
Updates one or more fields of a course work. See google.classroom.v1.CourseWork for details of which fields may be updated and who may change them. This request must be made by the Developer Console project of the [OAuth client ID](https://support.google.com/cloud/answer/6158849) used to create the corresponding course work item. This method returns the following error codes: * `PERMISSION_DENIED` if the requesting developer project did not create the corresponding course work, if the user is not permitted to make the requested modification to the student submission, or for access errors. * `INVALID_ARGUMENT` if the request is malformed. * `FAILED_PRECONDITION` if the requested course work has already been deleted. * `NOT_FOUND` if the requested course or course work does not exist.
@@ -630,7 +630,7 @@ Method Details
modifyAssignees(courseId, id, body=None, x__xgafv=None)
- Modifies assignee mode and options of a coursework. Only a teacher of the course that contains the coursework may call this method. This method returns the following error codes: * `PERMISSION_DENIED` if the requesting user is not permitted to access the requested course or course work or for access errors. * `INVALID_ARGUMENT` if the request is malformed. * `NOT_FOUND` if the requested course or course work does not exist.
+ Modifies assignee mode and options of a coursework. Only a teacher of the course that contains the coursework may call this method. This method returns the following error codes: * `PERMISSION_DENIED` if the requesting user is not permitted to access the requested course or course work or for access errors. * `INVALID_ARGUMENT` if the request is malformed. * `NOT_FOUND` if the requested course or course work does not exist. * `FAILED_PRECONDITION` for the following request error: * EmptyAssignees
Args:
courseId: string, Identifier of the course. This identifier can be either the Classroom-assigned identifier or an alias. (required)
diff --git a/docs/dyn/cloudbuild_v1.projects.builds.html b/docs/dyn/cloudbuild_v1.projects.builds.html
index ccfc3e054ca..c64407a5886 100644
--- a/docs/dyn/cloudbuild_v1.projects.builds.html
+++ b/docs/dyn/cloudbuild_v1.projects.builds.html
@@ -210,7 +210,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -624,7 +624,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -1069,7 +1069,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -1489,7 +1489,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
diff --git a/docs/dyn/cloudbuild_v1.projects.locations.builds.html b/docs/dyn/cloudbuild_v1.projects.locations.builds.html
index 3e230f77a99..e821d41095e 100644
--- a/docs/dyn/cloudbuild_v1.projects.locations.builds.html
+++ b/docs/dyn/cloudbuild_v1.projects.locations.builds.html
@@ -209,7 +209,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -623,7 +623,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -1068,7 +1068,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -1488,7 +1488,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
diff --git a/docs/dyn/cloudbuild_v1.projects.locations.triggers.html b/docs/dyn/cloudbuild_v1.projects.locations.triggers.html
index 36a0b388647..f7dbc864b69 100644
--- a/docs/dyn/cloudbuild_v1.projects.locations.triggers.html
+++ b/docs/dyn/cloudbuild_v1.projects.locations.triggers.html
@@ -198,7 +198,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -795,7 +795,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -1420,7 +1420,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -2029,7 +2029,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -2643,7 +2643,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -3242,7 +3242,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
diff --git a/docs/dyn/cloudbuild_v1.projects.triggers.html b/docs/dyn/cloudbuild_v1.projects.triggers.html
index 0fb97c33134..c8f0171c8c7 100644
--- a/docs/dyn/cloudbuild_v1.projects.triggers.html
+++ b/docs/dyn/cloudbuild_v1.projects.triggers.html
@@ -198,7 +198,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -795,7 +795,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -1420,7 +1420,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -2029,7 +2029,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -2644,7 +2644,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
@@ -3241,7 +3241,7 @@ Method Details
],
"npmPackages": [ # A list of npm packages to be uploaded to Artifact Registry upon successful completion of all build steps. Npm packages in the specified paths will be uploaded to the specified Artifact Registry repository using the builder service account's credentials. If any packages fail to be pushed, the build is marked FAILURE.
{ # Npm package to upload to Artifact Registry upon successful completion of all build steps.
- "packagePath": "A String", # Path to the package.json. e.g. workspace/path/to/package
+ "packagePath": "A String", # Optional. Path to the package.json. e.g. workspace/path/to/package Only one of `archive` or `package_path` can be specified.
"repository": "A String", # Artifact Registry repository, in the form "https://$REGION-npm.pkg.dev/$PROJECT/$REPOSITORY" Npm package in the workspace specified by path will be zipped and uploaded to Artifact Registry with this location as a prefix.
},
],
diff --git a/docs/dyn/cloudbuild_v2.projects.locations.html b/docs/dyn/cloudbuild_v2.projects.locations.html
index 61128fb3c0d..8a7e902b525 100644
--- a/docs/dyn/cloudbuild_v2.projects.locations.html
+++ b/docs/dyn/cloudbuild_v2.projects.locations.html
@@ -135,7 +135,7 @@ Method Details
Args:
name: string, The resource that owns the locations collection, if applicable. (required)
- extraLocationTypes: string, Optional. Do not use this field. It is unsupported and is ignored unless explicitly documented otherwise. This is primarily for internal usage. (repeated)
+ extraLocationTypes: string, Optional. Unless explicitly documented otherwise, don't use this unsupported field which is primarily intended for internal usage. (repeated)
filter: string, A filter to narrow down results to a preferred subset. The filtering language accepts strings like `"displayName=tokyo"`, and is documented in more detail in [AIP-160](https://google.aip.dev/160).
pageSize: integer, The maximum number of results to return. If not set, the service selects a default.
pageToken: string, A page token received from the `next_page_token` field in the response. Send that page token to receive the subsequent page.
diff --git a/docs/dyn/cloudcommerceprocurement_v1.html b/docs/dyn/cloudcommerceprocurement_v1.html
index 034452a4a5b..45c192ef4a6 100644
--- a/docs/dyn/cloudcommerceprocurement_v1.html
+++ b/docs/dyn/cloudcommerceprocurement_v1.html
@@ -95,17 +95,17 @@ Method Details
new_batch_http_request()
Create a BatchHttpRequest object based on the discovery document.
- Args:
- callback: callable, A callback to be called for each response, of the
- form callback(id, response, exception). The first parameter is the
- request id, and the second is the deserialized response object. The
- third is an apiclient.errors.HttpError exception object if an HTTP
- error occurred while processing the request, or None if no error
- occurred.
-
- Returns:
- A BatchHttpRequest object based on the discovery document.
-
+ Args:
+ callback: callable, A callback to be called for each response, of the
+ form callback(id, response, exception). The first parameter is the
+ request id, and the second is the deserialized response object. The
+ third is an apiclient.errors.HttpError exception object if an HTTP
+ error occurred while processing the request, or None if no error
+ occurred.
+
+ Returns:
+ A BatchHttpRequest object based on the discovery document.
+