Tabnine Logo
DefaultContainerExecutor.getPaths
Code IndexAdd Tabnine to your IDE (free)

How to use
getPaths
method
in
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor

Best Java code snippets using org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.getPaths (Showing top 4 results out of 315)

origin: io.hops/hadoop-yarn-server-nodemanager

@Private
@VisibleForTesting
protected ContainerLocalizer createContainerLocalizer(String user,
  String appId, String locId, List<String> localDirs,
  FileContext localizerFc, String userFolder) throws IOException {
 ContainerLocalizer localizer =
   new ContainerLocalizer(localizerFc, user, appId, locId,
     getPaths(localDirs),
     RecordFactoryProvider.getRecordFactory(getConf()), userFolder);
 return localizer;
}
origin: org.apache.hadoop/hadoop-yarn-server-nodemanager

/**
 * Create a new {@link ContainerLocalizer} instance.
 *
 * @param user the user who owns the job for which the localization is being
 * run
 * @param appId the ID of the application for which the localization is being
 * run
 * @param locId the ID of the container for which the localization is being
 * run
 * @param localDirs a list of directories to use as destinations for the
 * localization
 * @param localizerFc the {@link FileContext} to use when localizing files
 * @return the new {@link ContainerLocalizer} instance
 * @throws IOException if {@code user} or {@code locId} is {@code null} or if
 * the container localizer has an initialization failure
 */
@Private
@VisibleForTesting
protected ContainerLocalizer createContainerLocalizer(String user,
  String appId, String locId, List<String> localDirs,
  FileContext localizerFc) throws IOException {
 ContainerLocalizer localizer =
   new ContainerLocalizer(localizerFc, user, appId, locId,
     getPaths(localDirs),
     RecordFactoryProvider.getRecordFactory(getConf()));
 return localizer;
}
origin: ch.cern.hadoop/hadoop-yarn-server-nodemanager

@Override
public void startLocalizer(Path nmPrivateContainerTokensPath,
  InetSocketAddress nmAddr, String user, String appId, String locId,
  LocalDirsHandlerService dirsHandler)
  throws IOException, InterruptedException {
 List<String> localDirs = dirsHandler.getLocalDirs();
 List<String> logDirs = dirsHandler.getLogDirs();
 
 createUserLocalDirs(localDirs, user);
 createUserCacheDirs(localDirs, user);
 createAppDirs(localDirs, user, appId);
 createAppLogDirs(appId, logDirs, user);
 // randomly choose the local directory
 Path appStorageDir = getWorkingDir(localDirs, user, appId);
 String tokenFn = String.format(ContainerLocalizer.TOKEN_FILE_NAME_FMT, locId);
 Path tokenDst = new Path(appStorageDir, tokenFn);
 copyFile(nmPrivateContainerTokensPath, tokenDst, user);
 LOG.info("Copying from " + nmPrivateContainerTokensPath + " to " + tokenDst);
 FileContext localizerFc = FileContext.getFileContext(
   lfs.getDefaultFileSystem(), getConf());
 localizerFc.setUMask(lfs.getUMask());
 localizerFc.setWorkingDirectory(appStorageDir);
 LOG.info("Localizer CWD set to " + appStorageDir + " = " 
   + localizerFc.getWorkingDirectory());
 ContainerLocalizer localizer =
   new ContainerLocalizer(localizerFc, user, appId, locId, 
     getPaths(localDirs), RecordFactoryProvider.getRecordFactory(getConf()));
 // TODO: DO it over RPC for maintaining similarity?
 localizer.runLocalization(nmAddr);
}
origin: com.github.jiayuhan-it/hadoop-yarn-server-nodemanager

@Override
public void startLocalizer(Path nmPrivateContainerTokensPath,
  InetSocketAddress nmAddr, String user, String appId, String locId,
  LocalDirsHandlerService dirsHandler)
  throws IOException, InterruptedException {
 List<String> localDirs = dirsHandler.getLocalDirs();
 List<String> logDirs = dirsHandler.getLogDirs();
 
 createUserLocalDirs(localDirs, user);
 createUserCacheDirs(localDirs, user);
 createAppDirs(localDirs, user, appId);
 createAppLogDirs(appId, logDirs, user);
 // randomly choose the local directory
 Path appStorageDir = getWorkingDir(localDirs, user, appId);
 String tokenFn = String.format(ContainerLocalizer.TOKEN_FILE_NAME_FMT, locId);
 Path tokenDst = new Path(appStorageDir, tokenFn);
 copyFile(nmPrivateContainerTokensPath, tokenDst, user);
 LOG.info("Copying from " + nmPrivateContainerTokensPath + " to " + tokenDst);
 FileContext localizerFc = FileContext.getFileContext(
   lfs.getDefaultFileSystem(), getConf());
 localizerFc.setUMask(lfs.getUMask());
 localizerFc.setWorkingDirectory(appStorageDir);
 LOG.info("Localizer CWD set to " + appStorageDir + " = " 
   + localizerFc.getWorkingDirectory());
 ContainerLocalizer localizer =
   new ContainerLocalizer(localizerFc, user, appId, locId, 
     getPaths(localDirs), RecordFactoryProvider.getRecordFactory(getConf()));
 // TODO: DO it over RPC for maintaining similarity?
 localizer.runLocalization(nmAddr);
}
org.apache.hadoop.yarn.server.nodemanagerDefaultContainerExecutorgetPaths

Javadoc

Return the list of paths of given local directories.

Popular methods of DefaultContainerExecutor

  • buildCommandExecutor
    Create a new ShellCommandExecutor using the parameters.
  • containerIsAlive
    Returns true if the process with the specified pid is alive.
  • copyFile
    Copy a file using the #lfs FileContext.
  • createAppDirs
    Initialize the local directories for a particular user. * $local.dir/usercache/$user/appcache/$a
  • createAppLogDirs
    Create application log directories on all disks.
  • createContainerLogDirs
    Create application log directories on all disks.
  • createDir
    Use the #lfs FileContext to create the target directory.
  • createUserCacheDirs
    Initialize the local cache directories for a particular user. * $local.dir/usercache/$user *
  • createUserLocalDirs
    Initialize the local directories for a particular user. * * $local.dir/usercache/$user
  • getAppcacheDir
  • getApplicationDir
  • getConf
  • getApplicationDir,
  • getConf,
  • getDiskFreeSpace,
  • getFileCacheDir,
  • getLocalWrapperScriptBuilder,
  • getPidFilePath,
  • getRunCommand,
  • getUserCacheDir,
  • getWorkingDir

Popular in Java

  • Parsing JSON documents to java classes using gson
  • addToBackStack (FragmentTransaction)
  • setContentView (Activity)
  • getApplicationContext (Context)
  • UnknownHostException (java.net)
    Thrown when a hostname can not be resolved.
  • Path (java.nio.file)
  • SecureRandom (java.security)
    This class generates cryptographically secure pseudo-random numbers. It is best to invoke SecureRand
  • JLabel (javax.swing)
  • Runner (org.openjdk.jmh.runner)
  • LoggerFactory (org.slf4j)
    The LoggerFactory is a utility class producing Loggers for various logging APIs, most notably for lo
  • Top 12 Jupyter Notebook extensions
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now