medium
2 min read
Map Reduce
Map Reduce MapReduce is a programming paradigm for processing and generating large datasets. It involves two main steps: map and reduce. Map: The map...
Quick Actions
Insights
Difficultymedium
Reading Time2 min
Map Reduce MapReduce is a programming paradigm for processing and generating large datasets. It involves two main steps: map and reduce. Map: The map...
Map Reduce
MapReduce is a programming paradigm for processing and generating large datasets. It involves two main steps: map and reduce.
Map:
The map function iterates over the input dataset and transforms each element into a key-value pair.
The key is a function of the element, and the value can be any data type.
The map function can perform various operations, such as filtering, sorting, or data manipulation.
Reduce:
The reduce function aggregates the values associated with each key.
This can be done by performing a specific operation on the values, such as summing, counting, or averaging.
The reduce function can also perform more complex operations, such as grouping and summarizing.
Example:
Let's say we have a dataset of movie ratings, where each record has a movie ID, rating, and genre.
Map:
java
public void map(Object key, Object value) {
// Get movie ID from key
Integer movieId = (Integer) key;
// Get rating and genre from value
String rating = (String) value;
String genre = (String) value;
// Create key-value pair with movie ID and rating
key = movieId;
value = rating;
}
Reduce:
java
public void reduce(Object key, Iterable