Облачный поток данных Невозможно найти модель

Question

Я выполняю задание потока данных в облаке, которое выдает мне следующую ошибку

java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -156: Traceback (most recent call last):
  File "apache_beam/runners/common.py", line 813, in apache_beam.runners.common.DoFnRunner.process
  File "apache_beam/runners/common.py", line 610, in apache_beam.runners.common.PerWindowInvoker.invoke_process
  File "apache_beam/runners/common.py", line 685, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
  File "review_data_sim_pipeline_filter.py", line 47, in process
  File "review_data_sim_pipeline_filter.py", line 31, in predict_json
  File "/usr/local/lib/python3.5/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/googleapiclient/http.py", line 856, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://ml.googleapis.com/v1/projects/<myprojectname>/models/xgb_cloudml_train:predict?alt=json returned "Field: name Error: The model resource: "xgb_cloudml_train" was not found. Please create the Cloud ML model resource first by using 'gcloud ml-engine models create xgb_cloudml_train'.". Details: "[{'@type': 'type.googleapis.com/google.rpc.BadRequest', 'fieldViolations': [{'description': 'The model resource: "xgb_cloudml_train" was not found. Please create the Cloud ML model resource first by using \'gcloud ml-engine models create xgb_cloudml_train\'.', 'field': 'name'}]}]">

Однако моя модель действительно существует, и когда я использую код

def predict_json(self, project, model, instances, version=None): 
        import googleapiclient.discovery
        service = googleapiclient.discovery.build('ml', 'v1', discoveryServiceUrl='https://storage.googleapis.com/cloud-ml/discovery/ml_v1_discovery.json',cache_discovery=True)
        name = 'projects/{}/models/{}'.format(project, model)
        if version is not None:
            name += '/versions/{}'.format(version)
        response = service.projects().predict(
            name=name,
            body={'instances': instances}
        ).execute()
        if 'error' in response:
            raise RuntimeError(response['error'])
        return response['predictions']

, я получаю ответ , Однако, когда я использую с облаком мл, я получаю эту ошибку. Пожалуйста, помогите

Answer 1

На основании журналов, xgb_cloudml_train имеет активную и действительную версию модели по умолчанию? Я помню эту ошибку, когда версия не определена ...

https://ml.googleapis.com/v1/projects/<myprojectname>/models/xgb_cloudml_train:predict

В противном случае простой способ проверки URL с помощью curl для модели с версией не по умолчанию:

export PROJECT_ID=$(gcloud config get-value core/project 2> /dev/null)
export ACCESS_TOKEN=`gcloud auth print-access-token`
export MODEL_NAME="pretrained_model"
export MODEL_VERSION="cpu"

# Verify MODEL_NAME and MODEL_VERSION are defined
export URL=https://ml.googleapis.com/v1/projects/$PROJECT_ID/models/$MODEL_NAME/versions/$MODEL_VERSION:predict
## Send HTTPS Request
curl -X POST $URL -d @image_b64.json \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer $ACCESS_TOKEN" 

В случае версии по умолчанию вы можете использовать:

URL=https://ml.googleapis.com/v1/projects/$PROJECT_ID/models/$MODEL_NAME:predict

, что приведет к чему-то вроде этого:

https://ml.googleapis.com/v1/projects/dpe-cloud-mle/models/pretrained_model:predict

Этот пример кода Java может помочь проверить код DataFlow:

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.FileContent;
import com.google.api.client.http.GenericUrl;
import com.google.api.client.http.HttpContent;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestFactory;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.UriTemplate;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.discovery.Discovery;
import com.google.api.services.discovery.model.JsonSchema;
import com.google.api.services.discovery.model.RestDescription;
import com.google.api.services.discovery.model.RestMethod;
import java.io.File;

/*
 * Sample code for doing AI Platform online prediction in Java.
 */
public class OnlinePredictionSample {
  public static void main(String[] args) throws Exception {
    HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
    JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
    Discovery discovery = new Discovery.Builder(httpTransport, jsonFactory, null).build();

    RestDescription api = discovery.apis().getRest("ml", "v1").execute();
    RestMethod method = api.getResources().get("projects").getMethods().get("predict");

    JsonSchema param = new JsonSchema();
    String projectId = "YOUR_PROJECT_ID";
    // You should have already deployed a model and a version.
    // For reference, see https://cloud.google.com/ml-engine/docs/deploying-models.
    String modelId = "YOUR_MODEL_ID";
    String versionId = "YOUR_VERSION_ID";
    param.set(
        "name", String.format("projects/%s/models/%s/versions/%s", projectId, modelId, versionId));

    GenericUrl url =
        new GenericUrl(UriTemplate.expand(api.getBaseUrl() + method.getPath(), param, true));
    System.out.println(url);

    String contentType = "application/json";
    File requestBodyFile = new File("input.txt");
    HttpContent content = new FileContent(contentType, requestBodyFile);
    System.out.println(content.getLength());

    GoogleCredential credential = GoogleCredential.getApplicationDefault();
    HttpRequestFactory requestFactory = httpTransport.createRequestFactory(credential);
    HttpRequest request = requestFactory.buildRequest(method.getHttpMethod(), url, content);

    String response = request.execute().parseAsString();
    System.out.println(response);
  }
}

Кроме того, какую учетную запись вы используете для аутентификации через поток данных?