private JobStatus findJobForRunId(JobStatus[] jobs, ProgramRunId runId) throws NotFoundException { for (JobStatus job : jobs) { if (job.getJobName().startsWith(runId.getRun())) { return job; } } throw new NotFoundException(runId); }
private JobStatus findJobForRunId(JobStatus[] jobs, ProgramRunId runId) throws NotFoundException { for (JobStatus job : jobs) { if (job.getJobName().startsWith(runId.getRun())) { return job; } } throw new NotFoundException(runId); }
@Override public String toString() { return String.format("namespaceId=%s, applicationId=%s, program=%s, runid=%s", getNamespaceId(), getApplicationId(), getProgramName(), programRunId.getRun()); }
@Override public String toString() { return String.format("namespaceId=%s, applicationId=%s, program=%s, runid=%s", getNamespaceId(), getApplicationId(), getProgramName(), programRunId.getRun()); }
/** * @return run records for unfinished run */ private RunRecordMeta getUnfinishedRun(ProgramRunId programRunId) { MDSKey runningKey = getProgramRunInvertedTimeKey(TYPE_RUN_RECORD_ACTIVE, programRunId, RunIds.getTime(programRunId.getRun(), TimeUnit.SECONDS)); return get(runningKey, RunRecordMeta.class); }
private URL getWorkflowLocalDatasetURL(ProgramRunId workflowRunId) throws MalformedURLException { String path = String.format("apps/%s/workflows/%s/runs/%s/localdatasets", workflowRunId.getApplication(), workflowRunId.getProgram(), workflowRunId.getRun()); NamespaceId namespaceId = workflowRunId.getNamespaceId(); return config.resolveNamespacedURLV3(namespaceId, path); }
@Override protected void run() throws Exception { LOG.info("Starting workflow execution for '{}' with Run id '{}'", workflowSpec.getName(), workflowRunId.getRun()); LOG.trace("Workflow specification is {}", workflowSpec); workflowContext.setState(new ProgramState(ProgramStatus.RUNNING, null)); executeAll(workflowSpec.getNodes().iterator(), program.getApplicationSpecification(), new InstantiatorFactory(false), program.getClassLoader(), basicWorkflowToken); if (runningThread != null) { workflowContext.setState(new ProgramState(ProgramStatus.COMPLETED, null)); } LOG.info("Workflow '{}' with run id '{}' completed", workflowSpec.getName(), workflowRunId.getRun()); }
@Override protected void run() throws Exception { LOG.info("Starting workflow execution for '{}' with Run id '{}'", workflowSpec.getName(), workflowRunId.getRun()); LOG.trace("Workflow specification is {}", workflowSpec); workflowContext.setState(new ProgramState(ProgramStatus.RUNNING, null)); executeAll(workflowSpec.getNodes().iterator(), program.getApplicationSpecification(), new InstantiatorFactory(false), program.getClassLoader(), basicWorkflowToken); if (runningThread != null) { workflowContext.setState(new ProgramState(ProgramStatus.COMPLETED, null)); } LOG.info("Workflow '{}' with run id '{}' completed", workflowSpec.getName(), workflowRunId.getRun()); }
public DefaultProvisionerContext(ProgramRunId programRunId, Map<String, String> properties, SparkCompat sparkCompat, SSHContext sshContext) { this.programRun = new ProgramRun(programRunId.getNamespace(), programRunId.getApplication(), programRunId.getProgram(), programRunId.getRun()); this.properties = Collections.unmodifiableMap(new HashMap<>(properties)); this.sshContext = sshContext; this.sparkCompat = sparkCompat; this.cdapVersion = ProjectInfo.getVersion().toString(); }
public DefaultProvisionerContext(ProgramRunId programRunId, Map<String, String> properties, SparkCompat sparkCompat, SSHContext sshContext) { this.programRun = new ProgramRun(programRunId.getNamespace(), programRunId.getApplication(), programRunId.getProgram(), programRunId.getRun()); this.properties = Collections.unmodifiableMap(new HashMap<>(properties)); this.sshContext = sshContext; this.sparkCompat = sparkCompat; this.cdapVersion = ProjectInfo.getVersion().toString(); }
/** * @return The workflow token if the program is a workflow, {@code null} otherwise. */ @Nullable public WorkflowToken getWorkflowToken(ProgramRunId programRunId) { ProgramId programId = programRunId.getParent(); if (!programId.getType().equals(ProgramType.WORKFLOW)) { return null; } return store.getWorkflowToken(new WorkflowId(programId.getParent(), programId.getProgram()), programRunId.getRun()); }
/** * @return The workflow token if the program is a workflow, {@code null} otherwise. */ @Nullable public WorkflowToken getWorkflowToken(ProgramRunId programRunId) { ProgramId programId = programRunId.getParent(); if (!programId.getType().equals(ProgramType.WORKFLOW)) { return null; } return store.getWorkflowToken(new WorkflowId(programId.getParent(), programId.getProgram()), programRunId.getRun()); }
private Map<String, String> getMetricsTags(ProgramRunId programRunId, ProfileId profileId) { return ImmutableMap.<String, String>builder() .put(Constants.Metrics.Tag.PROFILE_SCOPE, profileId.getScope().name()) .put(Constants.Metrics.Tag.PROFILE, profileId.getProfile()) .put(Constants.Metrics.Tag.NAMESPACE, programRunId.getNamespace()) .put(Constants.Metrics.Tag.PROGRAM_TYPE, programRunId.getType().getPrettyName()) .put(Constants.Metrics.Tag.APP, programRunId.getApplication()) .put(Constants.Metrics.Tag.PROGRAM, programRunId.getProgram()) .put(Constants.Metrics.Tag.RUN_ID, programRunId.getRun()) .build(); }
private MDSKey getProgramRunInvertedTimeKey(String recordType, ProgramRunId runId, long startTs) { return getProgramKeyBuilder(recordType, runId.getParent()) .add(getInvertedTsKeyPart(startTs)) .add(runId.getRun()) .build(); }
private MDSKey getProgramRunInvertedTimeKey(String recordType, ProgramRunId runId, long startTs) { return getProgramKeyBuilder(recordType, runId.getParent()) .add(getInvertedTsKeyPart(startTs)) .add(runId.getRun()) .build(); }
private void setStartAndRunning(ProgramRunId id, Map<String, String> runtimeArgs, Map<String, String> systemArgs, ArtifactId artifactId) { long startTime = RunIds.getTime(id.getRun(), TimeUnit.SECONDS); setStart(id, runtimeArgs, systemArgs, artifactId); store.setRunning(id, startTime + 1, null, AppFabricTestHelper.createSourceId(++sourceId)); }
public DashboardProgramRunRecord(ProgramRunId runId, RunRecord runRecord, ArtifactId artifactId, String user, String startMethod) { this(runId.getNamespace(), ArtifactSummary.from(artifactId), new ApplicationNameVersion(runId.getApplication(), runId.getVersion()), runId.getType().name(), runId.getProgram(), runId.getRun(), user, startMethod, runRecord.getStartTs(), runRecord.getRunTs(), runRecord.getSuspendTs(), runRecord.getResumeTs(), runRecord.getStopTs(), runRecord.getStatus()); }
public DashboardProgramRunRecord(ProgramRunId runId, RunRecord runRecord, ArtifactId artifactId, String user, String startMethod) { this(runId.getNamespace(), ArtifactSummary.from(artifactId), new ApplicationNameVersion(runId.getApplication(), runId.getVersion()), runId.getType().name(), runId.getProgram(), runId.getRun(), user, startMethod, runRecord.getStartTs(), runRecord.getRunTs(), runRecord.getSuspendTs(), runRecord.getResumeTs(), runRecord.getStopTs(), runRecord.getStatus()); }
private MDSKey getKey(ProgramRunId programRunId) { return new MDSKey.Builder().add(KEY_PREFIX) .add(programRunId.getNamespace()) .add(programRunId.getApplication()) .add(programRunId.getVersion()) .add(programRunId.getType().name()) .add(programRunId.getProgram()) .add(programRunId.getRun()) .build(); }
private MDSKey getKey(ProgramRunId programRunId) { return new MDSKey.Builder().add(KEY_PREFIX) .add(programRunId.getNamespace()) .add(programRunId.getApplication()) .add(programRunId.getVersion()) .add(programRunId.getType().name()) .add(programRunId.getProgram()) .add(programRunId.getRun()) .build(); }