🐜
tiny engines
  • Tiny Engines
  • Personal Website Home
  • NFL machine learning capstone
    • project presentation
    • project proposal
    • project approach
    • project structure
    • project workflow
    • project summary
    • project code
  • Onboarding new hires
    • motivation
    • the project
    • the mailer service
      • mailer setup
      • walk-through
      • unit testing
      • testing the controller
      • testing the handler
      • testing the mailer
      • integration testing
      • integration example
      • acceptance testing
      • acceptance example
      • documenting the API
      • test coverage
      • performance testing
      • mutation testing
      • grammar checking
    • the event listener
      • design
      • webhook setup
      • walk-through
      • testing
      • the kafka connector
  • Walk-throughs
    • spark streaming hld
      • background
      • architecture
      • threat
      • project
      • transform-design
      • transform-poc
      • query-poc
    • kafka walkthroughs
    • java futures
      • async servers
      • async clients
      • async streams
Powered by GitBook
On this page

Was this helpful?

  1. Walk-throughs
  2. spark streaming hld

background

Previousspark streaming hldNextarchitecture

Last updated 3 years ago

Was this helpful?

AWS Simple Email Service sends emails to recipients on our behalf. For many organizations there's a lot more to sending emails to customers than just calling a sendmail program, and having a service lke SES manage that for us is a huge benefit. When SES sends emails for us, it also keeps track of whether those emails were delivered to the customer's inbox, and whether they were opened by the customer and even if the customer clicked on a link in the email.

Amazon SES can be configured to send event files back to us describing each milestone for a specific email. We'll get many events for each email -- SES sends an event when the email is:

Each event "class" comes in a separate stream and has a slightly different schema. We'll use Apache Spark to merge all the different event schemas into a single dataset that we can query to find the entire journey of any recipients email.