private static Map<String, String> getAdditionalMetadata(HiveTable table, Optional<HivePartition> partition, Optional<Exception> error) { ImmutableMap.Builder<String, String> builder = ImmutableMap.<String, String> builder().put(DB_NAME, table.getDbName()).put(TABLE_NAME, table.getTableName()); if (table.getLocation().isPresent()) { builder.put("Location", table.getLocation().get()); } if (partition.isPresent()) { builder.put("Partition", partition.get().toString()); } if (error.isPresent()) { builder.put(ERROR_MESSAGE, error.get().getMessage()); } return builder.build(); }
@Override public String toString() { String table = this.hiveSpec.getTable().getDbName() + "." + this.hiveSpec.getTable().getTableName(); String partitionInfo = this.hiveSpec.getPartition().isPresent() ? " partition " + Arrays.toString(this.hiveSpec.getPartition().get().getValues().toArray()) : ""; String location = this.hiveSpec.getPartition().isPresent() ? this.hiveSpec.getPartition().get().getLocation().get() : this.hiveSpec.getTable().getLocation().get(); return String.format("Register %s%s with location %s in Hive metastore %s.", table, partitionInfo, location, this.metastoreURI.isPresent() ? this.metastoreURI.get() : "default"); } }
@Override public void execute() throws IOException { if (this.verifyBeforeRegistering) { if (!this.hiveSpec.getTable().getLocation().isPresent()) { throw getException("Table does not have a location parameter."); } Path tablePath = new Path(this.hiveSpec.getTable().getLocation().get()); FileSystem fs = this.hiveSpec.getPath().getFileSystem(new Configuration()); if (!fs.exists(tablePath)) { throw getException(String.format("Table location %s does not exist.", tablePath)); } if (this.hiveSpec.getPartition().isPresent()) { if (!this.hiveSpec.getPartition().get().getLocation().isPresent()) { throw getException("Partition does not have a location parameter."); } Path partitionPath = new Path(this.hiveSpec.getPartition().get().getLocation().get()); if (!fs.exists(this.hiveSpec.getPath())) { throw getException(String.format("Partition location %s does not exist.", partitionPath)); } } } try (HiveRegister hiveRegister = HiveRegister.get(this.props, this.metastoreURI)) { log.info("Registering Hive Spec " + this.hiveSpec); ListenableFuture<Void> future = hiveRegister.register(this.hiveSpec); future.get(); } catch (InterruptedException | ExecutionException ie) { throw new IOException("Hive registration was interrupted.", ie); } }
private static Map<String, String> getAdditionalMetadata(HiveTable table, Optional<HivePartition> partition, Optional<Exception> error) { ImmutableMap.Builder<String, String> builder = ImmutableMap.<String, String> builder().put(DB_NAME, table.getDbName()).put(TABLE_NAME, table.getTableName()); if (table.getLocation().isPresent()) { builder.put("Location", table.getLocation().get()); } if (partition.isPresent()) { builder.put("Partition", partition.get().toString()); } if (error.isPresent()) { builder.put(ERROR_MESSAGE, error.get().getMessage()); } return builder.build(); }
@Override public String toString() { String table = this.hiveSpec.getTable().getDbName() + "." + this.hiveSpec.getTable().getTableName(); String partitionInfo = this.hiveSpec.getPartition().isPresent() ? " partition " + Arrays.toString(this.hiveSpec.getPartition().get().getValues().toArray()) : ""; String location = this.hiveSpec.getPartition().isPresent() ? this.hiveSpec.getPartition().get().getLocation().get() : this.hiveSpec.getTable().getLocation().get(); return String.format("Register %s%s with location %s in Hive metastore %s.", table, partitionInfo, location, this.metastoreURI.isPresent() ? this.metastoreURI.get() : "default"); } }
@Override public void execute() throws IOException { if (this.verifyBeforeRegistering) { if (!this.hiveSpec.getTable().getLocation().isPresent()) { throw getException("Table does not have a location parameter."); } Path tablePath = new Path(this.hiveSpec.getTable().getLocation().get()); FileSystem fs = this.hiveSpec.getPath().getFileSystem(new Configuration()); if (!fs.exists(tablePath)) { throw getException(String.format("Table location %s does not exist.", tablePath)); } if (this.hiveSpec.getPartition().isPresent()) { if (!this.hiveSpec.getPartition().get().getLocation().isPresent()) { throw getException("Partition does not have a location parameter."); } Path partitionPath = new Path(this.hiveSpec.getPartition().get().getLocation().get()); if (!fs.exists(this.hiveSpec.getPath())) { throw getException(String.format("Partition location %s does not exist.", partitionPath)); } } } try (HiveRegister hiveRegister = HiveRegister.get(this.props, this.metastoreURI)) { log.info("Registering Hive Spec " + this.hiveSpec); ListenableFuture<Void> future = hiveRegister.register(this.hiveSpec); future.get(); } catch (InterruptedException | ExecutionException ie) { throw new IOException("Hive registration was interrupted.", ie); } }