private void putIfNotUpdate( DbReadTable readTable, DbWriteTable writeTable, TableReference tableRef, List<Entry<Cell, Value>> batch, KeyAlreadyExistsException ex) { Map<Cell, Long> timestampByCell = Maps.newHashMap(); for (Entry<Cell, Value> entry : batch) { timestampByCell.put(entry.getKey(), entry.getValue().getTimestamp() + 1); } Map<Cell, Value> results = extractResults(readTable, tableRef, readTable.getLatestCells(timestampByCell, true)); ListIterator<Entry<Cell, Value>> iter = batch.listIterator(); while (iter.hasNext()) { Entry<Cell, Value> entry = iter.next(); Cell key = entry.getKey(); Value value = entry.getValue(); if (results.containsKey(key)) { if (results.get(key).equals(value)) { iter.remove(); } else { throw new KeyAlreadyExistsException( "primary key violation for key " + key + " with value " + value, ex); } } } writeTable.put(batch); }
private void putInternal( TableReference tableRef, Collection<Map.Entry<Cell, Value>> values, boolean doNotOverwriteWithSameValue) { Table table = getTableMap(tableRef); for (Map.Entry<Cell, Value> e : values) { byte[] contents = e.getValue().getContents(); long timestamp = e.getValue().getTimestamp(); Key key = getKey(table, e.getKey(), timestamp); byte[] oldContents = putIfAbsent(table, key, contents); if (oldContents != null && (doNotOverwriteWithSameValue || !Arrays.equals(oldContents, contents))) { throw new KeyAlreadyExistsException("We already have a value for this timestamp"); } } }
CheckAndSetResult casResult = checkAndSetRunner.executeCheckAndSet(client, request); if (!casResult.successful()) { return Optional.of(new KeyAlreadyExistsException( String.format("The row in table %s already exists.", tableRef.getQualifiedName()), ImmutableList.of(e.getKey())));
@Test public void loadShouldContinueIfKeyAlreadyExistsIsThrown() throws Exception { AtomicLong answerCount = new AtomicLong(); doAnswer((invocation) -> answerCount.get() > 0 ? VALID_COMMIT_TIMESTAMP : NO_TIMESTAMP) .when(mockTransactionService).get(VALID_START_TIMESTAMP); doAnswer((invocation) -> { answerCount.set(1); throw new KeyAlreadyExistsException("Already exists"); }).when(mockTransactionService).putUnlessExists(VALID_START_TIMESTAMP, ROLLBACK_TIMESTAMP); assertThat(loader.load(VALID_START_TIMESTAMP)).isEqualTo(VALID_COMMIT_TIMESTAMP); verify(mockTransactionService).putUnlessExists(VALID_START_TIMESTAMP, ROLLBACK_TIMESTAMP); }
private void put(List<Object[]> args) { try { String prefixedTableName = prefixedTableNames.get(tableRef, conns); conns.get().insertManyUnregisteredQuery("/* INSERT_ONE (" + prefixedTableName + ") */" + " INSERT INTO " + prefixedTableName + " (row_name, col_name, ts, val) " + " VALUES (?, ?, ?, ?) ", args); } catch (PalantirSqlException e) { if (ExceptionCheck.isUniqueConstraintViolation(e)) { throw new KeyAlreadyExistsException("primary key violation", e); } throw e; } }
@Test public void propagatesPutUnlessExistsExceptions() { KeyAlreadyExistsException exception = new KeyAlreadyExistsException("no"); doThrow(exception).when(delegate).putUnlessExists(anyLong(), anyLong()); assertThatThrownBy(() -> preStartHandlingService.putUnlessExists(START_TIMESTAMP, COMMIT_TIMESTAMP)) .isEqualTo(exception); verify(delegate).putUnlessExists(START_TIMESTAMP, COMMIT_TIMESTAMP); }
private void put(List<Object[]> args, List<Object[]> overflowArgs) { if (!overflowArgs.isEmpty()) { if (config.overflowMigrationState() == OverflowMigrationState.UNSTARTED) { conns.get().insertManyUnregisteredQuery("/* INSERT_OVERFLOW */" + " INSERT INTO " + config.singleOverflowTable() + " (id, val) VALUES (?, ?) ", overflowArgs); } else { String shortOverflowTableName = getShortOverflowTableName(); conns.get().insertManyUnregisteredQuery( "/* INSERT_OVERFLOW (" + shortOverflowTableName + ") */" + " INSERT INTO " + shortOverflowTableName + " (id, val) VALUES (?, ?) ", overflowArgs); } } try { String shortTableName = oraclePrefixedTableNames.get(tableRef, conns); conns.get().insertManyUnregisteredQuery("/* INSERT_ONE (" + shortTableName + ") */" + " INSERT INTO " + shortTableName + " (row_name, col_name, ts, val, overflow) " + " VALUES (?, ?, ?, ?, ?) ", args); } catch (PalantirSqlException e) { if (ExceptionCheck.isUniqueConstraintViolation(e)) { throw new KeyAlreadyExistsException("primary key violation", e); } throw e; } }
private void putBatch(DSLContext ctx, TableReference tableRef, PutBatch batch, boolean allowReinserts) { InsertValuesStep4<Record, byte[], byte[], Long, byte[]> query = ctx.insertInto(table(tableName(tableRef)), field(ROW_NAME, byte[].class), field(COL_NAME, byte[].class), field(TIMESTAMP, Long.class), field(VALUE, byte[].class)); query = batch.addValuesForInsert(query); try { query.execute(); } catch (DataAccessException e) { if (allowReinserts) { Result<? extends Record> records = ctx .select(A_ROW_NAME, A_COL_NAME, A_TIMESTAMP, A_VALUE) .from(atlasTable(tableRef).as(ATLAS_TABLE)) .where(row(A_ROW_NAME, A_COL_NAME, A_TIMESTAMP).in(batch.getRowsForSelect())) .fetch(); if (records.isEmpty()) { throw e; } PutBatch nextBatch = batch.getNextBatch(records); if (nextBatch != null) { putBatch(ctx, tableRef, nextBatch, allowReinserts); return; } } throw new KeyAlreadyExistsException("Conflict on table " + tableRef, e); } }
private void putInTransaction(final String tableName, final Map<Cell, byte[]> values, final long timestamp, final Handle handle) throws KeyAlreadyExistsException { try { batch(values.entrySet(), new Function<Collection<Entry<Cell, byte[]>>, Void>() { @Override @Nullable public Void apply(@Nullable final Collection<Entry<Cell, byte[]>> input) { putInternalInTransaction(tableName, input, timestamp, handle); return null; } }); } catch (RuntimeException e) { if (AtlasSqlUtils.isKeyAlreadyExistsException(e)) { throw new KeyAlreadyExistsException("Unique constraint violation", e); } throw e; } }
private void putInternal( TableReference tableRef, Collection<Map.Entry<Cell, Value>> values, boolean doNotOverwriteWithSameValue) { Table table = getTableMap(tableRef); for (Map.Entry<Cell, Value> e : values) { byte[] contents = e.getValue().getContents(); long timestamp = e.getValue().getTimestamp(); Key key = getKey(table, e.getKey(), timestamp); byte[] oldContents = putIfAbsent(table, key, contents); if (oldContents != null && (doNotOverwriteWithSameValue || !Arrays.equals(oldContents, contents))) { throw new KeyAlreadyExistsException("We already have a value for this timestamp"); } } }
private void putIfNotUpdate( DbReadTable readTable, DbWriteTable writeTable, TableReference tableRef, List<Entry<Cell, Value>> batch, KeyAlreadyExistsException ex) { Map<Cell, Long> timestampByCell = Maps.newHashMap(); for (Entry<Cell, Value> entry : batch) { timestampByCell.put(entry.getKey(), entry.getValue().getTimestamp() + 1); } Map<Cell, Value> results = extractResults(readTable, tableRef, readTable.getLatestCells(timestampByCell, true)); ListIterator<Entry<Cell, Value>> iter = batch.listIterator(); while (iter.hasNext()) { Entry<Cell, Value> entry = iter.next(); Cell key = entry.getKey(); Value value = entry.getValue(); if (results.containsKey(key)) { if (results.get(key).equals(value)) { iter.remove(); } else { throw new KeyAlreadyExistsException( "primary key violation for key " + key + " with value " + value, ex); } } } writeTable.put(batch); }
@Override public void putUnlessExists(final String tableName, final Map<Cell, byte[]> values) throws KeyAlreadyExistsException { try { batch(values.entrySet(), new Function<Collection<Entry<Cell, byte[]>>, Void>() { @Override @Nullable public Void apply(@Nullable final Collection<Entry<Cell, byte[]>> input) { getDbi().withHandle(new HandleCallback<Void>() { @Override public Void withHandle(Handle handle) throws Exception { putInternalInTransaction(tableName, input, 0L, handle); return null; } }); return null; } }); } catch (RuntimeException e) { if (AtlasSqlUtils.isKeyAlreadyExistsException(e)) { throw new KeyAlreadyExistsException("Unique constraint violation", e); } throw e; } }
@Override public void put(final String tableName, final Map<Cell, byte[]> values, final long timestamp) throws KeyAlreadyExistsException { try { batch(values.entrySet(), new Function<Collection<Entry<Cell, byte[]>>, Void>() { @Override @Nullable public Void apply(@Nullable final Collection<Entry<Cell, byte[]>> input) { getDbi().withHandle(new HandleCallback<Void>() { @Override public Void withHandle(Handle handle) throws Exception { deleteInternalInTransaction(tableName, Maps.transformValues(values, Functions.constant(timestamp)).entrySet(), handle); putInternalInTransaction(tableName, input, timestamp, handle); return null; } }); return null; } }); } catch (RuntimeException e) { if (AtlasSqlUtils.isKeyAlreadyExistsException(e)) { throw new KeyAlreadyExistsException("Unique constraint violation", e); } throw e; } }
CheckAndSetResult casResult = checkAndSetRunner.executeCheckAndSet(client, request); if (!casResult.successful()) { return Optional.of(new KeyAlreadyExistsException( String.format("The row in table %s already exists.", tableRef.getQualifiedName()), ImmutableList.of(e.getKey())));
private void put(List<Object[]> args) { try { String prefixedTableName = prefixedTableNames.get(tableRef, conns); conns.get().insertManyUnregisteredQuery("/* INSERT_ONE (" + prefixedTableName + ") */" + " INSERT INTO " + prefixedTableName + " (row_name, col_name, ts, val) " + " VALUES (?, ?, ?, ?) ", args); } catch (PalantirSqlException e) { if (ExceptionCheck.isUniqueConstraintViolation(e)) { throw new KeyAlreadyExistsException("primary key violation", e); } throw e; } }
@Override public void putUnlessExists(long startTimestamp, long commitTimestamp) throws KeyAlreadyExistsException { try { byte[] finalValue = proposer.propose(startTimestamp, PtBytes.toBytes(commitTimestamp)); long finalCommitTs = PtBytes.toLong(finalValue); try { // Make sure we do this put before we return because we want #get to succeed. kvStore.putAll(ImmutableMap.of(startTimestamp, finalCommitTs)); } catch (KeyAlreadyExistsException e) { // this case isn't worrisome } if (commitTimestamp != finalCommitTs) { throw new KeyAlreadyExistsException("Key " + startTimestamp + " already exists and is mapped to " + finalCommitTs); } } catch (PaxosRoundFailureException e) { throw new ServiceNotAvailableException("Could not store trascaction"); } }
@Override public void putUnlessExists(TableReference tableRef, Map<Cell, byte[]> values) throws KeyAlreadyExistsException { LockState<Cell> locks = lockSet.lockOnObjects(values.keySet()); try (Disposer d = new Disposer(); ColumnFamily table = columnFamilies.get(tableRef.getQualifiedName())) { Set<Cell> alreadyExists = Sets.newHashSetWithExpectedSize(0); WriteOptions options = d.register(new WriteOptions().setSync(writeOptions.fsyncCommit())); WriteBatch batch = d.register(new WriteBatch()); RocksIterator iter = d.register(getDb().newIterator(table.getHandle())); for (Entry<Cell, byte[]> entry : values.entrySet()) { byte[] key = RocksDbKeyValueServices.getKey(entry.getKey(), PUT_UNLESS_EXISTS_TS); if (RocksDbKeyValueServices.keyExists(iter, key)) { alreadyExists.add(entry.getKey()); } else { batch.put(table.getHandle(), key, entry.getValue()); } } getDb().write(options, batch); if (!alreadyExists.isEmpty()) { throw new KeyAlreadyExistsException("key already exists", alreadyExists); } } catch (RocksDBException e) { throw Throwables.propagate(e); } finally { locks.unlock(); } }
private void put(List<Object[]> args, List<Object[]> overflowArgs) { if (!overflowArgs.isEmpty()) { if (config.overflowMigrationState() == OverflowMigrationState.UNSTARTED) { conns.get().insertManyUnregisteredQuery("/* INSERT_OVERFLOW */" + " INSERT INTO " + config.singleOverflowTable() + " (id, val) VALUES (?, ?) ", overflowArgs); } else { String shortOverflowTableName = getShortOverflowTableName(); conns.get().insertManyUnregisteredQuery( "/* INSERT_OVERFLOW (" + shortOverflowTableName + ") */" + " INSERT INTO " + shortOverflowTableName + " (id, val) VALUES (?, ?) ", overflowArgs); } } try { String shortTableName = oraclePrefixedTableNames.get(tableRef, conns); conns.get().insertManyUnregisteredQuery("/* INSERT_ONE (" + shortTableName + ") */" + " INSERT INTO " + shortTableName + " (row_name, col_name, ts, val, overflow) " + " VALUES (?, ?, ?, ?, ?) ", args); } catch (PalantirSqlException e) { if (ExceptionCheck.isUniqueConstraintViolation(e)) { throw new KeyAlreadyExistsException("primary key violation", e); } throw e; } }