Co-authored-by: nahuhh
- jberman review
- cryptonote_protocol: don't arbitrarily download 1000 blocks ahead
- further restrict `proceed` to require `queue_proceed` in all cases.
ensure queue_proceed is true if we need the next block, even if we
already exceed the span and size limits
cryptonote_protocol: improved logging + const usage in span downloader
Set m_max_content_length to MAX_RPC_CONTENT_LENGTH (1MB) to match
the protection already implemented in core_rpc_server. This prevents
memory exhaustion attacks via unlimited HTTP request sizes.
Does the following to speedup the `/is_key_image_spent` RPC endpoint:
- Reads all on-chain key images in one LMDB read transacion
- Uses `cryptonote_core::are_key_images_spent_in_pool()` instead of `cryptonote_core::get_pool_transactions_and_spent_keys_info()` for pool querying. This only does a LMDB read per key image if in the pool.
- Filters known on-chain spent key images before querying for key images spent in pool
This RPC endpoint was causing major daemon slowdowns in the Carrot/FCMP++ Alpha Stressnet when using the `rescan_spent` command, especially for large wallets.
The effect was much worse if the mempool was full.
This replaces `ver_rct_non_semantics_simple_cached()` with an API that offloads
the responsibility of tracking input verification successes to the caller. The
main caller of this function in the codebase, `cryptonote::Blockchain()` instead
keeps track of the verification results for transaction in the mempool by
storing a "verification ID" in the mempool metadata table (with `txpool_tx_meta_t`).
This has several benefits, including:
* When the mempool is large (>8192 txs), we no longer experience cache misses and unnecessarily re-verify ring signatures. This greatly improves block propagation time for FCMP++ blocks under load
* For the same reason, reorg handling can be sped up by storing verification IDs of transactions popped from the chain
* Speeds up re-validating every mempool transaction on fork change (monerod revalidates the whole tx-pool on HFs #10142)
* Caches results for every single type of Monero transaction, not just latest RCT type
* Cache persists over a node restart
* Uses 512KiB less RAM (8192*2*32B)
* No additional storage or DB migration required since `txpool_tx_meta_t` already had padding allocated
* Moves more verification logic out of `cryptonote::Blockchain`
Furthermore, this opens the door to future multi-threaded block verification
speed-ups. Right now, transactions' input proof verification is limited to one
transaction at a time. However, one can imagine a scenario with verification IDs
where input proofs are optimistically multi-threaded in advance of block
processing. Then, even though ring member fetching and verification is
single-threaded inside of `cryptonote::Blockchain::check_tx_inputs()`, the
single thread can skip the CPU-intensive cryptographic code if the verification
ID allows it.
Also changes the default log category in `tx_verification_utils.cpp` from "blockchain" to "verify".
tobtoht