Question:I’ve been specifying the Avro data format through Kafka – but I’d like to understand the format in more detail and also understand why you would use Avro .

Answer: Avro is an Apache open source project which supplies data serialization. Avro supports exchanging data for Apache Hadoop. Avro is very useful in situations of different programs using different languages, exchanging data. When Avro serializes  , Avro stores the data definition and the data in one file.

Avro supports Schema evolution i.e data schemas changing over time is a key component of Avro.

Why would you use Avro ? Once you think about the Avro principles it make sense to consider Avro for certain circumstances.

> Data structures changing over time – aka rich data structures

>  Remote procedure calls – e.g Data transfers over a remote system and avoiding the java serialization overhead

> Persist data containers where the data schema and data do not need to be separated

> Avro binary output is smaller than Java , which supports a more efficient transfer ratio across remote procedure calls

Author: Jack Vamvas