From the course: Scala Essential Training for Data Science

Unlock this course with a free trial

Join today to access over 24,800 courses taught by industry experts.

Solution: Create a dataset

Solution: Create a dataset

From the course: Scala Essential Training for Data Science

Start my 1-month free trial Buy for my team

Solution: Create a dataset

“

(upbeat music) - [Narrator] Here's the solution. So I have started the Spark shell and I've imported Spark Implicits. First thing I want to do when I work with the dataset, I want to think about the schema. So I'm going to define a case class, and in this case, the case class is going to be location. And I have to have three attributes. An ID, a city and a country. Well, ID, I'm going to make int and city and country, I'll make string. So that's the definition for the case class location. And then we want to copy in that sequence definition that I included in the challenge. So I'll just go make a copy of that right now and paste that in. And so now we have our class definition, and we have a sequence of instances of those classes. So we actually have three instances. New York, London, and Paris. So the last piece of the equation is defining a dataset. And we'll use value for this. We're going to define a location, dataset, value. We're going to set the data type to be a dataset of…

Contents