Category: Reactive

Building a Simple REST API with Scala & Play! (Part 3)

In this 3 part series, we’ll cover creating a basic Play! REST API on top of Reactive Mongo. Full source code for this tutorial is available at https://github.com/jrodenbostel/getting-started-play-scala.

Welcome back!

We finished parts 1 & 2, which started with a description of the tools we’ll be using, and concluded with a fully functioning REST API built in Play! on top of a Reactive Mongo back-end. In part 3, we’ll cover the use of Spec2 and Mockito to write automated unit and integration tests for our application.

Integration Testing

Spec2 provides a DSL for BDD-style test specs. One variation of these specs, WithBrowser, actually runs the code in a headless browser, allowing you to end-to-end test your application in automated fashion. To get started, open the ‘/test/IntegrationSpec.scala’ file, which was created as part of our seed project. Update it to include the following:

import org.junit.runner.RunWith
import org.specs2.mutable.Specification
import org.specs2.runner.JUnitRunner
import play.api.test.WithBrowser

@RunWith(classOf[JUnitRunner])
class IntegrationSpec extends Specification {

  "Application" should {

    "work from within a browser" in new WithBrowser {

      browser.goTo("http://localhost:" + port)

      browser.pageSource must contain("Your database is ready.")
    }

    "remove data through the browser" in new WithBrowser {

      browser.goTo("http://localhost:" + port + "/cleanup")

      browser.pageSource must contain("Your database is clean.")
    }
  }
}

You’ll notice a spec for each Application Controller function, which simply visits the relevant URI and matches the response string. Tests can be executed from the Activator UI or by running ‘activator test’ from the command line from within your project folder.

Unit Testing

Unit testing is a bit more complex because we’ll test each controller by mocking the relevant repository method. Assertions for unit tests are more straightforward, as in most cases, we’re simply inspecting HTTP status codes. Update the ‘test/ApplicationSpec.scala’ file to include the following:

import controllers.{routes, Widgets}
import org.junit.runner.RunWith
import org.specs2.mock.Mockito
import org.specs2.mutable.Specification
import org.specs2.runner.JUnitRunner
import play.api.libs.json.{JsArray, Json}
import play.api.mvc.{Result, _}
import play.api.test.Helpers._
import play.api.test.{WithApplication, _}
import play.modules.reactivemongo.ReactiveMongoApi
import reactivemongo.api.commands.LastError
import reactivemongo.bson.BSONDocument
import repos.WidgetRepoImpl

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.{ExecutionContext, Future}

@RunWith(classOf[JUnitRunner])
class ApplicationSpec extends Specification with Results with Mockito {

  val mockRecipeRepo = mock[WidgetRepoImpl]
  val reactiveMongoApi = mock[ReactiveMongoApi]
  val documentId = "56a0ddb6c70000c700344254"
  val lastRequestStatus = new LastError(true, None, None, None, 0, None, false, None, None, false, None, None)

  val oatmealStout = Json.obj(
        "name" -> "Widget One",
        "description" -> "My first widget",
        "author" -> "Justin"
      )

  val posts = List(
    oatmealStout,
    Json.obj(
      "name" -> "Widget Two: The Return",
      "description" -> "My second widget",
      "author" -> "Justin"
    ))

  class TestController() extends Widgets(reactiveMongoApi) {
    override def widgetRepo: WidgetRepoImpl = mockRecipeRepo
  }

  val controller = new TestController()

  "Application" should {

    "send 404 on a bad request" in {
      new WithApplication() {
        route(FakeRequest(GET, "/boum")) must beSome.which(status(_) == NOT_FOUND)
      }
    }

    "Recipes#delete" should {
      "remove recipe" in {
        mockRecipeRepo.remove(any[BSONDocument])(any[ExecutionContext]) returns Future(lastRequestStatus)

        val result: Future[Result] = controller.delete(documentId).apply(FakeRequest())

        status(result) must be equalTo ACCEPTED
        there was one(mockRecipeRepo).remove(any[BSONDocument])(any[ExecutionContext])
      }
    }

    "Recipes#list" should {
      "list recipes" in {
        mockRecipeRepo.find()(any[ExecutionContext]) returns Future(posts)

        val result: Future[Result] = controller.index().apply(FakeRequest())

        contentAsJson(result) must be equalTo JsArray(posts)
        there was one(mockRecipeRepo).find()(any[ExecutionContext])
      }
    }

    "Recipes#read" should {
      "read recipe" in {
        mockRecipeRepo.select(any[BSONDocument])(any[ExecutionContext]) returns Future(Option(oatmealStout))

        val result: Future[Result] = controller.read(documentId).apply(FakeRequest())

        contentAsJson(result) must be equalTo oatmealStout
        there was one(mockRecipeRepo).select(any[BSONDocument])(any[ExecutionContext])
      }
    }

    "Recipes#create" should {
      "create recipe" in {
        mockRecipeRepo.save(any[BSONDocument])(any[ExecutionContext]) returns Future(lastRequestStatus)

        val request = FakeRequest().withBody(oatmealStout)
        val result: Future[Result] = controller.create()(request)

        status(result) must be equalTo CREATED
        there was one(mockRecipeRepo).save(any[BSONDocument])(any[ExecutionContext])
      }
    }

    "Recipes#update" should {
      "update recipe" in {
        mockRecipeRepo.update(any[BSONDocument], any[BSONDocument])(any[ExecutionContext]) returns Future(lastRequestStatus)

        val request = FakeRequest().withBody(oatmealStout)
        val result: Future[Result] = controller.update(documentId)(request)

        status(result) must be equalTo ACCEPTED
        there was one(mockRecipeRepo).update(any[BSONDocument], any[BSONDocument])(any[ExecutionContext])
      }
    }
  }
}

You’ll notice the class starts with the creation of Mocks and example data – this is very straightforward and was pulled from examples of real BSON data from Mongo. In each spec, you’ll also notice the same pattern: mocks are configured, methods are exercised, and assertions are made. Very similar to other BDD-style test frameworks like Jasmine and Rspec.

Conclusion

Reactive programming and related frameworks, especially those of the functional variety, represent a significant shift in the way we write applications. There are many new patterns, tools, and programming styles that a developer must become familiar with in order to write applications in the reactive style effectively. Many of us, myself included, are just starting to get opportunities to do so. Although these tools may not be appropriate for every solution, becoming familiar with the underlying concepts will help individuals become more well-rounded as developers and help ensure that scalable, resilient, responsive architectures are adopted when necessary.

Building a Simple REST API with Scala & Play! (Part 2)

In this 3 part series, we’ll cover creating a basic Play! REST API on top of Reactive Mongo.  Full source code for this tutorial is available at https://github.com/jrodenbostel/getting-started-play-scala.

Welcome back!

If you’re coming in fresh, and need some instructions on getting the appropriate tools installed and creating a shell of an environment, please refer to part 1.  In part 2, we’ll cover adding our first API functions as asynchronous actions in a Play! controller, as well as define our first data access functions.
Because we started with a seed project, we got some bonus cruft in our application in the form of a views package.  While Scala templates are a fine way to create views for your application, for this tutorial, we’ll be building a RESTful API that responds with JSON strings. Start by deleting the ‘views’ folder at ‘app/views’ such that:
Screen Shot 2016-01-25 at 10.03.14 PM
Note that this will break the default controller that came as part of the project seed.  To remedy this, update the default controller action response to simply render text, by replacing:
Ok(views.html.index("Your new application is ready."))
with:
Ok("Your new application is ready.")

Creating the controller

Next, we’ll create our controller.  Create a new file in the ‘app/controllers’ folder named ‘Widgets.scala’.  This will house the RESTful actions associated with Widgets.  We’ll also add default methods to this controller for the RESTful actions that we’ll implement later.
package controllers

import play.api.mvc._

class Widgets extends Controller {

  def index = TODO

  def create = TODO

  def read(id: String) = TODO

  def update(id: String) = TODO

  def delete(id: String) = TODO
}
Play! comes with a default “TODO” page for controller actions that have not yet been implemented.  It’s a nice way to keep your app functioning and reloading while you’re building out functionality incrementally.  Before we can see that default “TODO” page, we must first add routes to the application configuration that reference our new controller and it’s not-yet-implemented actions.
Update /conf/routes with paths to the Widgets controller such that the following is true:
# Routes
# This file defines all application routes (Higher priority routes first)
# ~~~~

# Home page
GET        /                    controllers.Application.index
GET        /cleanup             controllers.Application.cleanup

#Widgets
GET        /api/widgets         controllers.Widgets.index
GET        /api/widget/:id      controllers.Widgets.read(id: String)
POST       /api/widget          controllers.Widgets.create
DELETE     /api/widget/:id      controllers.Widgets.delete(id: String)
PATCH      /api/widget/:id      controllers.Widgets.update(id: String)
Now, we can visit one of these new paths to view the default “TODO” screen.
Screen Shot 2016-01-25 at 10.18.35 PM

Data access

Before we can build out our controller actions, we’ll add a data access layer to surface some data to our controller.  Let’s start by creating the trait, which will define the contract for our data access layer.  (Traits are similar to Interfaces in Java – more info that here http://docs.scala-lang.org/tutorials/tour/traits.html)
Create a new folder called ‘repos’ such that a directory at ‘app/repos’ exists.  In that directory, create a new Scala file named WidgetRepo.scala with the following content:
app/repos/WidgetRepo.scala:
package repos

import javax.inject.Inject

import play.api.libs.json.{JsObject, Json}
import play.modules.reactivemongo.ReactiveMongoApi
import play.modules.reactivemongo.json._
import play.modules.reactivemongo.json.collection.JSONCollection
import reactivemongo.api.ReadPreference
import reactivemongo.api.commands.WriteResult
import reactivemongo.bson.{BSONDocument, BSONObjectID}

import scala.concurrent.{ExecutionContext, Future}

trait WidgetRepo {
  def find()(implicit ec: ExecutionContext): Future[List[JsObject]]

  def select(selector: BSONDocument)(implicit ec: ExecutionContext): Future[Option[JsObject]]

  def update(selector: BSONDocument, update: BSONDocument)(implicit ec: ExecutionContext): Future[WriteResult]

  def remove(document: BSONDocument)(implicit ec: ExecutionContext): Future[WriteResult]

  def save(document: BSONDocument)(implicit ec: ExecutionContext): Future[WriteResult]
}
There should be no surprises there – normal CRUD operations for data access and manipulation.  Things you may notice that are special here – the return types.  They’re all asynchronous Futures.  The read operations return JsObjects https://www.playframework.com/documentation/2.0/api/scala/play/api/libs/json/JsObject.html and the write operations return WriteResults http://reactivemongo.org/releases/0.11/documentation/tutorial/write-documents.html.
Let’s continue by adding the implementations for each of these methods in the form of a Scala class in the same source file:
app/repos/WidgetRepo.scala:
class WidgetRepoImpl @Inject() (reactiveMongoApi: ReactiveMongoApi) extends WidgetRepo {

  def collection = reactiveMongoApi.db.collection[JSONCollection]("widgets");

  override def find()(implicit ec: ExecutionContext): Future[List[JsObject]] = {
    val genericQueryBuilder = collection.find(Json.obj());
    val cursor = genericQueryBuilder.cursor[JsObject](ReadPreference.Primary);
    cursor.collect[List]()
  }

  override def select(selector: BSONDocument)(implicit ec: ExecutionContext): Future[Option[JsObject]] = {
    collection.find(selector).one[JsObject]
  }

  override def update(selector: BSONDocument, update: BSONDocument)(implicit ec: ExecutionContext): Future[WriteResult] = {
    collection.update(selector, update)
  }

  override def remove(document: BSONDocument)(implicit ec: ExecutionContext): Future[WriteResult] = {
    collection.remove(document)
  }

  override def save(document: BSONDocument)(implicit ec: ExecutionContext): Future[WriteResult] = {
    collection.update(BSONDocument("_id" -> document.get("_id").getOrElse(BSONObjectID.generate)), document, upsert = true)
  }

}

Again, there shouldn’t be much that’s surprising here other than the normal, somewhat complex IMO, Scala syntax.  You can see the implicit ExecutionContext, which, for asynchronous code, basically let’s Scala decide where in the thread pool to execute the related function.  You may also notice the ReadPreference in the find() function.  This tells Mongo that our repo would like to read it’s results from the primary Mongo node.

Back to the controller

At this point, we can return to our controller to round out the implementation details there.  Let’s start simple.
Before we can start adding implementation details, we need to configure some dependency injection details.  We’ll inject the ReactiveMongoApi into our Controller, then use that to create our repository.  Update the signature of the Widgets controller and imports so the following is true:
package controllers

import javax.inject.Inject

import play.api.libs.concurrent.Execution.Implicits.defaultContext
import play.api.libs.json.Json
import play.api.mvc._
import play.modules.reactivemongo.{MongoController, ReactiveMongoApi, ReactiveMongoComponents}
import reactivemongo.api.commands.WriteResult
import reactivemongo.bson.{BSONObjectID, BSONDocument}
import repos.WidgetRepoImpl

class Widgets @Inject()(val reactiveMongoApi: ReactiveMongoApi) extends Controller
    with MongoController with ReactiveMongoComponents {

  def widgetRepo = new WidgetRepoImpl(reactiveMongoApi)

In ‘app/controllers/Widgets.scala’, we have an unimplemented ‘index’ function.  This is meant to display a list of all widgets in the database, selected without parameters.  The implementation for this function is very straightforward. Update the index function such that:

  def index = Action.async { implicit request =>
    widgetRepo.find().map(widgets => Ok(Json.toJson(widgets)))
  }
In this implementation, since we’re expecting a single result, when we execute ‘map’ on the result, we’ll take the resulting object and render it directly to our JSON response.
The ‘read’ method, which is very similar to ‘index’, will take a single String parameter (the id of the document being searched for) and return a single result as JSON string. Update the read function such that:
  def read(id: String) = Action.async { implicit request =>
    widgetRepo.select(BSONDocument(Id -> BSONObjectID(id))).map(widget => Ok(Json.toJson(widget)))
  }

 The ‘delete’ method is similarly straightforward in that it takes a String id for the document to be deleted, and returns a HTTP status code 202 (Accepted), and no body.

  def delete(id: String) = Action.async {
    widgetRepo.remove(BSONDocument(Id -> BSONObjectID(id)))
        .map(result => Accepted)
  }

The ‘create’ and ‘update’ methods introduce a small amount of complexity in that they require the request body to be parsed using Scala’s built in pattern matching functionality. Since we’ll have to match the field names in two places, we’ll create a companion object to hold our field names. Create a ‘WidgetFields’ companion object in the Widgets controller source file:

object WidgetFields {
  val Id = "_id"
  val Name ="name"
  val Description = "description"
  val Author = "author"
}

In the body of the Widget controller, add a scoped import for the companion object:

import controllers.WidgetFields._

For the ‘create’ method, we’ll apply a JSON Body Parser to an implicit request, parsing out the relevant content needed to build a BSONDocument that can be persisted via the Repo:

def create = Action.async(BodyParsers.parse.json) { implicit request =>
    val name = (request.body \ Name).as[String]
    val description = (request.body \ Description).as[String]
    val author = (request.body \ Author).as[String]
    recipeRepo.save(BSONDocument(
      Name -> name,
      Description -> description,
      Author -> author
    )).map(result => Created)
  }

Execution of this method returns the HTTP status code 201.  For the ‘update’ method, we’ll perform largely the same operation – applying the JSON Body Parser to an implicit request, however, this time we’ll call a different repo method – this time building a BSONDocument to select the relevant document with, then passing in the current field values:

  def update(id: String) = Action.async(BodyParsers.parse.json) { implicit request =>
    val name = (request.body \ Name).as[String]
    val description = (request.body \ Description).as[String]
    val author = (request.body \ Author).as[String]
    widgetRepo.update(BSONDocument(Id -> BSONObjectID(id)),
      BSONDocument("$set" -> BSONDocument(Name -> name, Description -> description, Author -> author)))
        .map(result => Accepted)
  }

Testing!

Part 3 of this series will cover testing using the Spec2 library. In the mean time…We have a fully functioning REST API – but testing manually requires configuration and the execution of HTTP operations. Many web frameworks are packed with functionality that allows a developer to ‘bootstrap’ an application – adding seed data to a local environment for testing as an example. Recent changes in the Play! framework’s GlobalSettings have changed the way developers do things like seed test databases (https://www.playframework.com/documentation/2.4.x/GlobalSettings). While the dust settles, and while we wait for part 3, I created some helper functions in the Application controller that will create and remove some test data:

package controllers

import javax.inject.{Inject, Singleton}

import play.api.Logger
import play.api.libs.concurrent.Execution.Implicits.defaultContext
import play.api.libs.json.Json
import play.api.mvc.{Action, Controller}
import play.modules.reactivemongo.json.collection.JSONCollection
import play.modules.reactivemongo.{MongoController, ReactiveMongoApi, ReactiveMongoComponents}
import reactivemongo.api.collections.bson.BSONCollection
import reactivemongo.api.commands.bson.BSONCountCommand.{ Count, CountResult }
import reactivemongo.api.commands.bson.BSONCountCommandImplicits._
import reactivemongo.bson.BSONDocument

import scala.concurrent.Future

@Singleton
class Application @Inject()(val reactiveMongoApi: ReactiveMongoApi) extends Controller
    with MongoController with ReactiveMongoComponents {

  def jsonCollection = reactiveMongoApi.db.collection[JSONCollection]("widgets");
  def bsonCollection = reactiveMongoApi.db.collection[BSONCollection]("widgets");

  def index = Action {
    Logger.info("Application startup...")

    val posts = List(
      Json.obj(
        "name" -> "Widget One",
        "description" -> "My first widget",
        "author" -> "Justin"
      ),
      Json.obj(
        "name" -> "Widget Two: The Return",
        "description" -> "My second widget",
        "author" -> "Justin"
      ))

    val query = BSONDocument("name" -> BSONDocument("$exists" -> true))
    val command = Count(query)
    val result: Future[CountResult] = bsonCollection.runCommand(command)

    result.map { res =>
      val numberOfDocs: Int = res.value
      if(numberOfDocs > 1) {
        jsonCollection.bulkInsert(posts.toStream, ordered = true).foreach(i => Logger.info("Record added."))
      }
    }

    Ok("Your database is ready.")
  }

  def cleanup = Action {
    jsonCollection.drop().onComplete {
      case _ => Logger.info("Database collection dropped")
    }
    Ok("Your database is clean.")
  }
}

… and routes …

# Home page
GET        /                    controllers.Application.index
GET        /cleanup             controllers.Application.cleanup




	


	



			
				


			

			
Building a Simple REST API with Scala & Play! (Part 1)
		
	
	

		
In this 3 part series, we’ll cover creating a basic Play! REST API on top of Reactive Mongo.  Full source code for this tutorial is available at https://github.com/jrodenbostel/getting-started-play-scala.






Why Scala & Play!?


There has been a lot of buzz recently in the industry and at our some of our clients around Reactive Programming and related frameworks.   Reactive Programming is a movement based around building applications that can meet the diverse demands of modern environments.  The four main characteristics are:




    

  • Responsiveness:  high-performance, with consistent and fast response times
    • 

  • Resilience: fails gracefully, and remains responsive during failures
    • 

  • Elasticity: remains responsive during varying workload
    • 

  • Message Driven: non-blocking, back-pressure capable, asynchronous
    • 





Moves towards cloud infrastructure, microservices architecture, and DevOps-style tools which have made conveniences of previously cumbersome tasks, all lend themselves to supporting reactive systems: systems composed of many small pieces, possibly distributed, expanding and contracting based on load, location agnostic, relying on messaging to communicate.  Maybe you’ve seen parts of this and haven’t realized it has a name.  Systems designed based on these principles can be considered reactive.






Like many things in development, learning a new tool or technique often requires not only reading, but also coding a live example. In this blog series, I’ll cover creating a small, simple reactive system – a simple RESTful API using the Play! Framework and Reactive Mongo.  Because reactive programming is often associated with functional programming, I’ll also be writing this example in Scala.






The Play! Framework (https://playframework.com) is an open source web app framework that’s been around since 2007.  In many ways, it’s similar to other web application frameworks you may be familiar with like Spring MVC and Rails: it’s MVC-based, it comes with a lot of built-in support tooling (scaffolding, execution, dependency management, etc), and it’s based on the principle of convention over configuration.  In other ways, mostly ways that indicate how it fits into the world of reactive systems, it differs from those frameworks: it’s 100% stateless and is built to run on Netty (http://netty.io) – a non-blocking, asynchronous application framework.






Behind Play!, Reactive Mongo (http://reactivemongo.org) will give us non-blocking and asynchronous access to a Mongo document store through a Scala driver and a Play! module for easy integration into a Play! apps.  The Reactive Mongo API exposes most normal data access functions you’d come to expect, but returns results as Scala Futures, and provides translation utilities for translating the Mongo document format (BSON) to JSON, and many functional helper methods for dealing with result sets.






To wrap it all up, we’ll be using Spec2 (https://etorreborre.github.io/specs2/) to unit and integration test our application.  Spec2 allows us to write test cases in the style of behavior-driven development and highlights the flexibility of Scala and how it an easily be used to create a domain-specific language.






You may find the tools in this tutorial more difficult to get started with than others you may be used to – there are many new concepts in the mix here – it’s to be expected.  These tools have a place in a creating highly-available, fault-tolerant systems capable of handling web-scale traffic.  If you were actually building what we’ll build in this tutorial, this may not be the right tool set.






The Setup




To get started, make sure you have Scala installed.  There are many ways to install Scala.  My favorite is by using the Homebrew package manager available for the Mac (http://brewformulas.org/Scala). You can also download, unpack binaries, and update paths by following the instructions here (http://www.scala-lang.org/download/install.html).






You’re also going to want to have Mongo installed.  Again, this is something that I normally install using the Homebrew package manager.  More detailed instructions can be found here: https://docs.mongodb.org/manual/tutorial/install-mongodb-on-os-x/.  You should obviously have Mongo running during this tutorial.






Assuming Scala is installed, next download and install the Typesafe Activator, available here (https://www.typesafe.com/activator/download).  This will give us a nice UI for generating our application, running and viewing the results of tests, and running our application.






After installing the Typesafe Activator, open a command line prompt and start it up:




Justins-MacBook-Pro:Projects justin$ activator ui







From the Typesafe Activator window, under the ‘Templates’ menu item, select ‘Seeds’, then select ‘Play Scala Seed’.  Don’t forget to give your application a name and location before hitting the ‘Create app’ button.




Screen Shot 2016-01-25 at 9.33.33 PM






After pressing the ‘Create app’ button, you should be greeted with a message indicating that your application was created successfully.  From this window, we’ll be able to start/stop, compile, and test our new application.  Remember, Play! supports hot-swapping of code, so we’ll be doing a lot of viewing results in this window.






 Screen Shot 2016-01-25 at 9.29.27 PM




Installing Dependencies


Play! applications generated from the Play! Scala seed that we just used come packed with a pre-defined build script written using SBT.  SBT is the de-facto build tool for Scala applications from Typesafe.  More information on SBT can be found here (http://www.scala-sbt.org). Our new application has a build.sbt file that we’ll need to update with a dependency for Reactive Mongo.  Update the library dependencies sequence in build.sbt accordingly:






libraryDependencies ++= Seq(
  jdbc,
  cache,
  ws,
  specs2 % Test,
  "org.reactivemongo" %% "play2-reactivemongo" % "0.11.7.play24"
)







Much like Maven or Bundler, this will automatically download and install the Reactive Mongo Play! module, which will in turn download the necessary dependent Reactive Mongo and Mongo libraries.






Next, we’ll update our application.conf file to include configuration information about our Mongo instance.  The application.conf file is found at /conf/application.conf and contains generally configuration settings for your application.  We have two lines to add to this file.  Add the following at the end of application.conf and save your changes:






play.modules.enabled += "play.modules.reactivemongo.ReactiveMongoModule"

mongodb.uri = "mongodb://localhost:27017/getting-started-play-scala"







At this point it’s probably worth noting that if you’re interested in exploring or manipulating a Mongo instance, I recommend using Robomongo (http://robomongo.org).






To conclude part 1, using the Typesafe Activator, run your application.  If we’ve installed our dependencies correctly, we should be greated with the default Welcome to Play screen, as seen below:






Screen Shot 2016-01-26 at 8.31.54 PM.png






Please continue to part 2, where we’ll begin to define our REST API by creating a Play! controller with asynchronous actions, and then move on to creating our data access layer using Reactive Mongo.