RisingWave provides a streaming data warehouse to power AI-based search products for Kaito, a leading web3 information platform indexing content not accessible via traditional search engines. Crypto traders and investors need solid, real-time data to identify trends, possible fraud, and nuanced risk. However, much of the data to feed that need is found within resources that traditional search engines simply do not index, such as blockchains and private information sources. Kaito, a fintech company, has taken on this challenge to provide the crypto industry with an authoritative crypto data access engine with powerful AI capabilities to guide informed decision making.
To do this, Kaito needs to incorporate real-time stream processing rather than traditional batch processing that also minimizes or eliminates the nuanced challenges of stream processing. Data freshness: second level number of data pipelines: Several hundred number of supported cryptos: Several thousand. Initially, Kaito sought a stream processing system to read and transform data from upstream sources then forward the results to a downstream system responsible for ‘materializing’ (multiple sources married into a single source) the data and handling user queries. However, introducing a second system presented challenges such as any changes to queries in one system must be automatically mirrored in the other to meet evolving business needs; data transfer between systems can become bottlenecked leading to additional serialization-deserialization and network costs; syncing upstream to downstream systems requires engineers to reread the data causing latency and increased costs; and, importantly, when multiple streaming jobs sync data into different tables in the downstream system, maintaining consistency between tables is difficult.
RisingWave distinguishes itself—and became the stream processing solution of choice for Kaito—by combining stream processing and server queries within a single system. As a result, users in RisingWave can define two types of streaming jobs. ‘Create sink’ continuously processes input data and updates the downstream system. ‘Create materialized view’ continuously reflects updates in a materialized view so that all views accurately reflect changes brought about by the same volume of upstream data.
Additionally, RisingWave is PostgreSQL-compatible which reduces the learning curve so that Kaito can speed up the development process and onboard new user with a low learning curve. RisingWave also offers Java, Python, and Rust User-Defined Functions (UDFs) to handle the 5% of cases where SQL may fall short. Furthermore, third-party tools and client libraries can seamlessly connect to RisingWave as if it were PostSQL, which is crucial for Kaito’s reliance on a variety of managed services and open-source tools to develop products. In short, RisingWave offers Kaito a stream processing solution that is a critical collaborator to its unique value proposition.
The force-multiplier for RisingWave is its partnership with Amazon Web Services (AWS) AWS provides robust services for storage, compute, and networking, enabling RisingWave to deliver low-latency, high-performance real-time analytics at scale. The partnership also allows RisingWave to integrate seamlessly with the AWS ecosystem, giving customers a streamlined deployment experience and access to a wide range of cloud-native tools. Ultimately, this partnership helps RisingWave ensure it can elastically scale to meet real-time processing needs for organizations of all sizes, meet high standards of security and compliance via AWS enterprise-grade security framework, and deliver seamless streaming data experience through native connections to AWS resources such as Amazon Managed Streaming for Apache Kafka and Amazon Redshift.
Delivering Continuous, Real-Time Stream Processing for Kaito. In all, Kaito utilizes RisingWave as the streaming data warehouse to power its AI-based search platform for crypto traders and investors as well as its user-facing analytical dashboards, internal operational workloads, and real-time alerts among other applications. By consolidating its real-time data infrastructure for various applications into a single system (e.g., RisingWave), Kaito significantly reduces processing costs and enhances data freshness; provides analytics and operational insights to clients and the in-house operations team with near-instantaneous latency; drastically lowers development costs and shortens the learning curve as engineers can focus exclusively on implementing new features using Postgres-compatible SQL and leveraging existing ecosystems.
RisingWave, based in San Francisco, delivers fresh, low-latency insights from real-time streams, database CDC, and time-series data. With support from the AWS Global Startup Program, it brings streaming and batch together, letting users join and analyze both live and historical data, and persist results in managed Apache Iceberg tables. The mission of the company is to empower every organization to build real-time, event-driven applications with simplicity, scalability, and cost-effectiveness. By leveraging technologies like Apache Iceberg and offering features such as compute isolation and cross-database querying, RisingWave aims to support enterprises in developing and deploying real-time applications efficiently and at scale.
Kaito, Seattle, Washington, based, is a next generation web3 information platform that indexes a wide range of web3 content that is not easily accessible through traditional search engines. This includes sources like social media, governance forums, research, news, podcasts, conference transcripts, and more. By leveraging advanced AI technologies, Kaito redefines how users discover and interact with blockchain-related information. Our enterprise platform empowers users across the web3 ecosystem, offering revolutionary information access and data-driven insights. http://www.kaito.ai
Leave a Reply