### Questions 1. How to handle errors in logging the data? Type error or so on? How will we be notified? RBDMS will throw error in sentry 2. What are the data points? 3. How to retain 1 year old data and use it? 4. How to add new data? 5. How to make breaking changes to the data stored for a better version of it? 6. What is the overhead (ms) in using DB over ES. 7. No updates to the call information?
Consider this scenario, you get a call for order refund status in e-commerce domain we raise a concerned ticket in the e-commerce platform and keep the call details as unresolved. We need to mark this as resolved by the client either by contacting the customer manually or in an automated way. 8. Multi tenancy ? Separate database for each client? Possible categories of data - Write only data. - Significant/Critical data. - Not so significant data. - Structured data. - Unstructured data. Possible Architectural Solutions 1. To store call logs in RDBMS - Straight forward approach. 2. To push details to a message queue - Consume and store in RDBMS, to avoid overhead time of storing in DB. 3. Push details to a message queue - Consume and store in NOSQL like Mongo, Cassandra. 4. Push and forget all the structured data directly to ES. 5. Use application logs and store in ES using beats or fluentd. 6. Keep the data model generic and just use the stucts To build Activity dashboard and query the data. — RDBMS — Logging approach RESEARCH ES - Primary data store Good option for cases with only writes (no updates), many reads and wherein there is no need for transactions, integrity, constraints (datatype, PK, FK, NOT NULL, DEFAULT, UNIQUE), correctness and robustness Elasticsearch is commonly used in addition to another database.