Tabnine Logo
ExternalPayloadStorage$PayloadType
Code IndexAdd Tabnine to your IDE (free)

How to use
ExternalPayloadStorage$PayloadType
in
com.netflix.conductor.common.utils

Best Java code snippets using com.netflix.conductor.common.utils.ExternalPayloadStorage$PayloadType (Showing top 18 results out of 315)

origin: Netflix/conductor

Monitors.recordExternalPayloadStorageUsage(workflow.getWorkflowName(), ExternalPayloadStorage.Operation.READ.toString(), ExternalPayloadStorage.PayloadType.WORKFLOW_INPUT.toString());
workflowInstance.setInput(workflowInputParams);
workflowInstance.setExternalInputPayloadStoragePath(null);
    if (StringUtils.isNotBlank(task.getExternalOutputPayloadStoragePath())) {
      task.setOutputData(externalPayloadStorageUtils.downloadPayload(task.getExternalOutputPayloadStoragePath()));
      Monitors.recordExternalPayloadStorageUsage(task.getTaskDefName(), ExternalPayloadStorage.Operation.READ.toString(), ExternalPayloadStorage.PayloadType.TASK_OUTPUT.toString());
      task.setExternalOutputPayloadStoragePath(null);
      Monitors.recordExternalPayloadStorageUsage(task.getTaskDefName(), ExternalPayloadStorage.Operation.READ.toString(), ExternalPayloadStorage.PayloadType.TASK_INPUT.toString());
      task.setExternalInputPayloadStoragePath(null);
origin: Netflix/conductor

/**
 * Uses the {@link PayloadStorage} for storing large payloads.
 * Gets the uri for storing the payload from the server and then uploads to this location.
 *
 * @param payloadType  the {@link com.netflix.conductor.common.utils.ExternalPayloadStorage.PayloadType} to be uploaded
 * @param payloadBytes the byte array containing the payload
 * @param payloadSize  the size of the payload
 * @return the path where the payload is stored in external storage
 */
protected String uploadToExternalPayloadStorage(ExternalPayloadStorage.PayloadType payloadType, byte[] payloadBytes, long payloadSize) {
  Preconditions.checkArgument(payloadType.equals(ExternalPayloadStorage.PayloadType.WORKFLOW_INPUT) || payloadType.equals(ExternalPayloadStorage.PayloadType.TASK_OUTPUT),
      "Payload type must be workflow input or task output");
  ExternalStorageLocation externalStorageLocation = payloadStorage.getLocation(ExternalPayloadStorage.Operation.WRITE, payloadType, "");
  payloadStorage.upload(externalStorageLocation.getUri(), new ByteArrayInputStream(payloadBytes), payloadSize);
  return externalStorageLocation.getPath();
}
origin: Netflix/conductor

  taskResult.setOutputData(null);
} else {
  WorkflowTaskMetrics.incrementExternalPayloadUsedCount(taskType, ExternalPayloadStorage.Operation.WRITE.name(), ExternalPayloadStorage.PayloadType.TASK_OUTPUT.name());
  String externalStoragePath = uploadToExternalPayloadStorage(ExternalPayloadStorage.PayloadType.TASK_OUTPUT, taskOutputBytes, taskResultSize);
  taskResult.setExternalOutputPayloadStoragePath(externalStoragePath);
origin: com.netflix.conductor/conductor-core

  ((Task) entity).setInputData(null);
  ((Task) entity).setExternalInputPayloadStoragePath(uploadHelper(payloadBytes, payloadSize, PayloadType.TASK_INPUT));
  Monitors.recordExternalPayloadStorageUsage(((Task) entity).getTaskDefName(), ExternalPayloadStorage.Operation.WRITE.toString(), PayloadType.TASK_INPUT.toString());
  break;
case TASK_OUTPUT:
  ((Task) entity).setOutputData(null);
  ((Task) entity).setExternalOutputPayloadStoragePath(uploadHelper(payloadBytes, payloadSize, PayloadType.TASK_OUTPUT));
  Monitors.recordExternalPayloadStorageUsage(((Task) entity).getTaskDefName(), ExternalPayloadStorage.Operation.WRITE.toString(), PayloadType.TASK_OUTPUT.toString());
  break;
case WORKFLOW_INPUT:
  ((Workflow) entity).setInput(null);
  ((Workflow) entity).setExternalInputPayloadStoragePath(uploadHelper(payloadBytes, payloadSize, PayloadType.WORKFLOW_INPUT));
  Monitors.recordExternalPayloadStorageUsage(((Workflow) entity).getWorkflowType(), ExternalPayloadStorage.Operation.WRITE.toString(), PayloadType.WORKFLOW_INPUT.toString());
  break;
case WORKFLOW_OUTPUT:
  ((Workflow) entity).setOutput(null);
  ((Workflow) entity).setExternalOutputPayloadStoragePath(uploadHelper(payloadBytes, payloadSize, PayloadType.WORKFLOW_OUTPUT));
  Monitors.recordExternalPayloadStorageUsage(((Workflow) entity).getWorkflowType(), ExternalPayloadStorage.Operation.WRITE.toString(), PayloadType.WORKFLOW_OUTPUT.toString());
  break;
origin: com.netflix.conductor/conductor-client

  taskResult.setOutputData(null);
} else {
  WorkflowTaskMetrics.incrementExternalPayloadUsedCount(taskType, ExternalPayloadStorage.Operation.WRITE.name(), ExternalPayloadStorage.PayloadType.TASK_OUTPUT.name());
  String externalStoragePath = uploadToExternalPayloadStorage(ExternalPayloadStorage.PayloadType.TASK_OUTPUT, taskOutputBytes, taskResultSize);
  taskResult.setExternalOutputPayloadStoragePath(externalStoragePath);
origin: com.netflix.conductor/conductor-client

  throw new ConductorClientException(errorMsg);
} else {
  WorkflowTaskMetrics.incrementExternalPayloadUsedCount(startWorkflowRequest.getName(), ExternalPayloadStorage.Operation.WRITE.name(), ExternalPayloadStorage.PayloadType.WORKFLOW_INPUT.name());
  String externalStoragePath = uploadToExternalPayloadStorage(ExternalPayloadStorage.PayloadType.WORKFLOW_INPUT, workflowInputBytes, workflowInputSize);
  startWorkflowRequest.setExternalInputPayloadStoragePath(externalStoragePath);
origin: com.netflix.conductor/conductor-core

/**
 * Updates the workflow output.
 *
 * @param workflow the workflow instance
 * @param task     if not null, the output of this task will be copied to workflow output if no output parameters are specified in the workflow defintion
 *                 if null, the output of the last task in the workflow will be copied to workflow output of no output parameters are specified in the workflow definition
 */
void updateWorkflowOutput(final Workflow workflow, @Nullable Task task) {
  List<Task> allTasks = workflow.getTasks();
  if (allTasks.isEmpty()) {
    return;
  }
  Task last = Optional.ofNullable(task).orElse(allTasks.get(allTasks.size() - 1));
  WorkflowDef workflowDef = workflow.getWorkflowDefinition();
  Map<String, Object> output;
  if (workflowDef.getOutputParameters() != null && !workflowDef.getOutputParameters().isEmpty()) {
    Workflow workflowInstance = populateWorkflowAndTaskData(workflow);
    output = parametersUtils.getTaskInput(workflowDef.getOutputParameters(), workflowInstance, null, null);
  } else if (StringUtils.isNotBlank(last.getExternalOutputPayloadStoragePath())) {
    output = externalPayloadStorageUtils.downloadPayload(last.getExternalOutputPayloadStoragePath());
    Monitors.recordExternalPayloadStorageUsage(last.getTaskDefName(), ExternalPayloadStorage.Operation.READ.toString(), ExternalPayloadStorage.PayloadType.TASK_OUTPUT.toString());
  } else {
    output = last.getOutputData();
  }
  workflow.setOutput(output);
  externalPayloadStorageUtils.verifyAndUpload(workflow, ExternalPayloadStorage.PayloadType.WORKFLOW_OUTPUT);
}
origin: com.netflix.conductor/conductor-core

Monitors.recordExternalPayloadStorageUsage(workflow.getWorkflowName(), ExternalPayloadStorage.Operation.READ.toString(), ExternalPayloadStorage.PayloadType.WORKFLOW_INPUT.toString());
workflowInstance.setInput(workflowInputParams);
workflowInstance.setExternalInputPayloadStoragePath(null);
    if (StringUtils.isNotBlank(task.getExternalOutputPayloadStoragePath())) {
      task.setOutputData(externalPayloadStorageUtils.downloadPayload(task.getExternalOutputPayloadStoragePath()));
      Monitors.recordExternalPayloadStorageUsage(task.getTaskDefName(), ExternalPayloadStorage.Operation.READ.toString(), ExternalPayloadStorage.PayloadType.TASK_OUTPUT.toString());
      task.setExternalOutputPayloadStoragePath(null);
      Monitors.recordExternalPayloadStorageUsage(task.getTaskDefName(), ExternalPayloadStorage.Operation.READ.toString(), ExternalPayloadStorage.PayloadType.TASK_INPUT.toString());
      task.setExternalInputPayloadStoragePath(null);
origin: com.netflix.conductor/conductor-client

/**
 * This method is not intended to be used in the client.
 * The client makes a request to the server to get the {@link ExternalStorageLocation}
 */
@Override
public ExternalStorageLocation getLocation(Operation operation, PayloadType payloadType, String path) {
  String uri;
  switch (payloadType) {
    case WORKFLOW_INPUT:
    case WORKFLOW_OUTPUT:
      uri = "workflow";
      break;
    case TASK_INPUT:
    case TASK_OUTPUT:
      uri = "tasks";
      break;
    default:
      throw new ConductorClientException(String.format("Invalid payload type: %s for operation: %s", payloadType.toString(), operation.toString()));
  }
  return clientBase.getForEntity(String.format("%s/externalstoragelocation", uri), new Object[]{"path", path}, ExternalStorageLocation.class);
}
origin: com.netflix.conductor/conductor-client

/**
 * Uses the {@link PayloadStorage} for storing large payloads.
 * Gets the uri for storing the payload from the server and then uploads to this location.
 *
 * @param payloadType  the {@link com.netflix.conductor.common.utils.ExternalPayloadStorage.PayloadType} to be uploaded
 * @param payloadBytes the byte array containing the payload
 * @param payloadSize  the size of the payload
 * @return the path where the payload is stored in external storage
 */
protected String uploadToExternalPayloadStorage(ExternalPayloadStorage.PayloadType payloadType, byte[] payloadBytes, long payloadSize) {
  Preconditions.checkArgument(payloadType.equals(ExternalPayloadStorage.PayloadType.WORKFLOW_INPUT) || payloadType.equals(ExternalPayloadStorage.PayloadType.TASK_OUTPUT),
      "Payload type must be workflow input or task output");
  ExternalStorageLocation externalStorageLocation = payloadStorage.getLocation(ExternalPayloadStorage.Operation.WRITE, payloadType, "");
  payloadStorage.upload(externalStorageLocation.getUri(), new ByteArrayInputStream(payloadBytes), payloadSize);
  return externalStorageLocation.getPath();
}
origin: com.netflix.conductor/conductor-client

/**
 * Populates the workflow output from external payload storage if the external storage path is specified.
 *
 * @param workflow the workflow for which the output is to be populated.
 */
private void populateWorkflowOutput(Workflow workflow) {
  if (StringUtils.isNotBlank(workflow.getExternalOutputPayloadStoragePath())) {
    WorkflowTaskMetrics.incrementExternalPayloadUsedCount(workflow.getWorkflowName(), ExternalPayloadStorage.Operation.READ.name(), ExternalPayloadStorage.PayloadType.WORKFLOW_OUTPUT.name());
    workflow.setOutput(downloadFromExternalStorage(ExternalPayloadStorage.PayloadType.WORKFLOW_OUTPUT, workflow.getExternalOutputPayloadStoragePath()));
  }
}
origin: com.netflix.conductor/conductor-client

/**
 * Populates the task input from external payload storage if the external storage path is specified.
 *
 * @param task the task for which the input is to be populated.
 */
private void populateTaskInput(Task task) {
  if (StringUtils.isNotBlank(task.getExternalInputPayloadStoragePath())) {
    WorkflowTaskMetrics.incrementExternalPayloadUsedCount(task.getTaskDefName(), ExternalPayloadStorage.Operation.READ.name(), ExternalPayloadStorage.PayloadType.TASK_INPUT.name());
    task.setInputData(downloadFromExternalStorage(ExternalPayloadStorage.PayloadType.TASK_INPUT, task.getExternalInputPayloadStoragePath()));
    task.setExternalInputPayloadStoragePath(null);
  }
}
origin: Netflix/conductor

  ((Task) entity).setInputData(null);
  ((Task) entity).setExternalInputPayloadStoragePath(uploadHelper(payloadBytes, payloadSize, PayloadType.TASK_INPUT));
  Monitors.recordExternalPayloadStorageUsage(((Task) entity).getTaskDefName(), ExternalPayloadStorage.Operation.WRITE.toString(), PayloadType.TASK_INPUT.toString());
  break;
case TASK_OUTPUT:
  ((Task) entity).setOutputData(null);
  ((Task) entity).setExternalOutputPayloadStoragePath(uploadHelper(payloadBytes, payloadSize, PayloadType.TASK_OUTPUT));
  Monitors.recordExternalPayloadStorageUsage(((Task) entity).getTaskDefName(), ExternalPayloadStorage.Operation.WRITE.toString(), PayloadType.TASK_OUTPUT.toString());
  break;
case WORKFLOW_INPUT:
  ((Workflow) entity).setInput(null);
  ((Workflow) entity).setExternalInputPayloadStoragePath(uploadHelper(payloadBytes, payloadSize, PayloadType.WORKFLOW_INPUT));
  Monitors.recordExternalPayloadStorageUsage(((Workflow) entity).getWorkflowType(), ExternalPayloadStorage.Operation.WRITE.toString(), PayloadType.WORKFLOW_INPUT.toString());
  break;
case WORKFLOW_OUTPUT:
  ((Workflow) entity).setOutput(null);
  ((Workflow) entity).setExternalOutputPayloadStoragePath(uploadHelper(payloadBytes, payloadSize, PayloadType.WORKFLOW_OUTPUT));
  Monitors.recordExternalPayloadStorageUsage(((Workflow) entity).getWorkflowType(), ExternalPayloadStorage.Operation.WRITE.toString(), PayloadType.WORKFLOW_OUTPUT.toString());
  break;
origin: Netflix/conductor

  throw new ConductorClientException(errorMsg);
} else {
  WorkflowTaskMetrics.incrementExternalPayloadUsedCount(startWorkflowRequest.getName(), ExternalPayloadStorage.Operation.WRITE.name(), ExternalPayloadStorage.PayloadType.WORKFLOW_INPUT.name());
  String externalStoragePath = uploadToExternalPayloadStorage(ExternalPayloadStorage.PayloadType.WORKFLOW_INPUT, workflowInputBytes, workflowInputSize);
  startWorkflowRequest.setExternalInputPayloadStoragePath(externalStoragePath);
origin: Netflix/conductor

/**
 * Updates the workflow output.
 *
 * @param workflow the workflow instance
 * @param task     if not null, the output of this task will be copied to workflow output if no output parameters are specified in the workflow defintion
 *                 if null, the output of the last task in the workflow will be copied to workflow output of no output parameters are specified in the workflow definition
 */
void updateWorkflowOutput(final Workflow workflow, @Nullable Task task) {
  List<Task> allTasks = workflow.getTasks();
  if (allTasks.isEmpty()) {
    return;
  }
  Task last = Optional.ofNullable(task).orElse(allTasks.get(allTasks.size() - 1));
  WorkflowDef workflowDef = workflow.getWorkflowDefinition();
  Map<String, Object> output;
  if (workflowDef.getOutputParameters() != null && !workflowDef.getOutputParameters().isEmpty()) {
    Workflow workflowInstance = populateWorkflowAndTaskData(workflow);
    output = parametersUtils.getTaskInput(workflowDef.getOutputParameters(), workflowInstance, null, null);
  } else if (StringUtils.isNotBlank(last.getExternalOutputPayloadStoragePath())) {
    output = externalPayloadStorageUtils.downloadPayload(last.getExternalOutputPayloadStoragePath());
    Monitors.recordExternalPayloadStorageUsage(last.getTaskDefName(), ExternalPayloadStorage.Operation.READ.toString(), ExternalPayloadStorage.PayloadType.TASK_OUTPUT.toString());
  } else {
    output = last.getOutputData();
  }
  workflow.setOutput(output);
  externalPayloadStorageUtils.verifyAndUpload(workflow, ExternalPayloadStorage.PayloadType.WORKFLOW_OUTPUT);
}
origin: Netflix/conductor

/**
 * This method is not intended to be used in the client.
 * The client makes a request to the server to get the {@link ExternalStorageLocation}
 */
@Override
public ExternalStorageLocation getLocation(Operation operation, PayloadType payloadType, String path) {
  String uri;
  switch (payloadType) {
    case WORKFLOW_INPUT:
    case WORKFLOW_OUTPUT:
      uri = "workflow";
      break;
    case TASK_INPUT:
    case TASK_OUTPUT:
      uri = "tasks";
      break;
    default:
      throw new ConductorClientException(String.format("Invalid payload type: %s for operation: %s", payloadType.toString(), operation.toString()));
  }
  return clientBase.getForEntity(String.format("%s/externalstoragelocation", uri), new Object[]{"path", path}, ExternalStorageLocation.class);
}
origin: Netflix/conductor

/**
 * Populates the task input from external payload storage if the external storage path is specified.
 *
 * @param task the task for which the input is to be populated.
 */
private void populateTaskInput(Task task) {
  if (StringUtils.isNotBlank(task.getExternalInputPayloadStoragePath())) {
    WorkflowTaskMetrics.incrementExternalPayloadUsedCount(task.getTaskDefName(), ExternalPayloadStorage.Operation.READ.name(), ExternalPayloadStorage.PayloadType.TASK_INPUT.name());
    task.setInputData(downloadFromExternalStorage(ExternalPayloadStorage.PayloadType.TASK_INPUT, task.getExternalInputPayloadStoragePath()));
    task.setExternalInputPayloadStoragePath(null);
  }
}
origin: Netflix/conductor

/**
 * Populates the workflow output from external payload storage if the external storage path is specified.
 *
 * @param workflow the workflow for which the output is to be populated.
 */
private void populateWorkflowOutput(Workflow workflow) {
  if (StringUtils.isNotBlank(workflow.getExternalOutputPayloadStoragePath())) {
    WorkflowTaskMetrics.incrementExternalPayloadUsedCount(workflow.getWorkflowName(), ExternalPayloadStorage.Operation.READ.name(), ExternalPayloadStorage.PayloadType.WORKFLOW_OUTPUT.name());
    workflow.setOutput(downloadFromExternalStorage(ExternalPayloadStorage.PayloadType.WORKFLOW_OUTPUT, workflow.getExternalOutputPayloadStoragePath()));
  }
}
com.netflix.conductor.common.utilsExternalPayloadStorage$PayloadType

Most used methods

  • toString
  • equals
  • name

Popular in Java

  • Making http requests using okhttp
  • onRequestPermissionsResult (Fragment)
  • scheduleAtFixedRate (Timer)
  • compareTo (BigDecimal)
  • Menu (java.awt)
  • IOException (java.io)
    Signals a general, I/O-related error. Error details may be specified when calling the constructor, a
  • SocketException (java.net)
    This SocketException may be thrown during socket creation or setting options, and is the superclass
  • TimeZone (java.util)
    TimeZone represents a time zone offset, and also figures out daylight savings. Typically, you get a
  • JList (javax.swing)
  • FileUtils (org.apache.commons.io)
    General file manipulation utilities. Facilities are provided in the following areas: * writing to a
  • Github Copilot alternatives
Tabnine Logo

  • Products

    Search for Java codeSearch for JavaScript code

  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode

  • Company

    About UsContact UsCareers

  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now