Over the years at Capital One, the Spark framework evolved into the technology of choice for high volume real-time streaming and batch needs. After noticing some pains that come with operating the Spark infrastructure, the team decided to find a simpler, low maintenance yet highly scalable pattern and designed a Serverless Streaming solution around it: Apache Spark. Ever wonder what it was like to wait for a monthly statement to arrive to find any erroneous transactions on your account?
Maybe you lived through it and still have the papercut scars to prove it. The digital revolution has made it possible to access your information when and where you need it and we are not too far from a futuristic world where relevant information comes to you as it happens and you don’t need to go looking out for it. Capital One is on a journey to make almost everything we do real time, adapting cloud computing and big data engineering tools to bring a real time aspect to our solutions.
Source: capitalone.com