Build LMS platform from scratch. Part 6 — How I Structure API Development

Hi folks, with each step we are closer to something looking more like a product than a prototype. I am happy to see you here and in this part, we will cover vital parts of the API feature development.

There are three elephants on which we will build each part of our API:

  • Business logic module (Here we will define functions that are related primarily to the business and which do not belong to any application details).

  • Controller module (Here we expose our business logic to the world via routes in our API).

  • Tests module (Source of truth and primary place for knowing that our application works as expected. Each module will be tested at least via one method: unit or integration test).

Along with those leading parts, there will be additional parts such as database table migrations, policy for authorized access for each exposed functionality, and specific changes. So I will take our user business logic from the previous article and build a complete API including authorization.

Business logic module

Recalling what we’ve done previously [see previous article], we can remember the user creation function:

defmodule Backend.Users do

  #... other functions
  
  def create_user(attrs) do
    attrs
    |> Backend.Users.Schema.User.create_changeset()
    |> Backend.Repo.insert()
  end
end

Let’s reuse it in the authentication module via simple delegation to the user’s module. Also, I have added a function that can check whether the provided user credentials are valid in our system:

defmodule Backend.Auth do
  defdelegate register(attrs), to: Backend.Users, as: :create_user

  def verify_user_credentials(email, password) do
    user = Backend.Repo.get_by(Backend.Users.Schema.User, email: email)

    case user do
      nil ->
        {:error, "Invalid email or password"}

      user ->
        case Argon2.verify_pass(password, user.password_hash) do
          true ->
            {:ok, user}

          false ->
            {:error, "Invalid email or password"}
        end
    end
  end
end

Notice that the newly created authentication module is pretty simple, exposing only two main functions: register/1 and verify_user_credentials/2. But in the future, it will grow and become larger, which is why we’re already grouping everything related to authentication into a separate place and calling it a context module — a place for cohesive, determined, and encapsulated data. (See https://hexdocs.pm/phoenix/contexts.html). The benefits from such an approach are enormous:

  • Keep the web layer (controllers) clean and thin.

  • Switch the logic and source of data for each context module independently. For example, If I want to authenticate users via a separate service or load specific modules from a different database it is possible with such separation of the concerns.

  • Reusability between different places of usage (Controllers, Background Jobs, CLI)

  • Ability to write unit tests without the need to involve controllers and full infrastructure.

Controllers module

Since we can register and verify user identities, we’re ready to build authorization API for the external world. Let’s define such a controller with two functions:

defmodule BackendWeb.AuthController do
  use BackendWeb, :controller

  def login(conn, %{"email" => email, "password" => password}) do
    with {:ok, user} <- Backend.Auth.verify_user_credentials(email, password) do
      conn
      |> put_session(:current_user, user)
      |> successful_response(%{user: user})
    else
      {:error, message} ->
        conn |> unauthorized_response(message)
    end
  end

  def register(conn, params) do
    with {:ok, user} <- Backend.Auth.register(params) do
      conn
      |> put_session(:current_user, user)
      |> successful_response(%{user: user})
    else
      {:error, changeset} ->
        conn |> failed_changeset_response(changeset)
    end
  end
end

Notice how we cover our authentication module with the internals of request logic. We are checking whether we have appropriate data for requests and saving users’ authentication tokens in the cookies (Plug.Session module) for future needs via put_session/3 function, otherwise — we are throwing errors via helper functions.

Sessions

When you bootstrap a Phoenix template it already has built-in sessions functionality provided by the Plug.Session module. The only two things that you need to implement to make it work:

  1. Configure it via settings options.

  2. Decide under what conditions session keys should be assigned, then implement that logic.

In my case, the endpoint which serves my webserver has such settings:

defmodule BackendWeb.Endpoint
  use Phoenix.Endpoint, otp_app: :backend
   
  # ...

  # The session will be stored in the cookie and signed,
  # this means its contents can be read but not tampered with.
  @session_options [
    store: :cookie,
    key: "_backend_key",
    signing_salt: "vrH2jxaB",
    same_site: "Lax",
    # 1 day in seconds
    max_age: 60 * 60 * 24
  ]
  
  plug Plug.Session, @session_options
  
  # ...
end

When the put_session/3 function is invoked with some data (e.g., a user), it automatically stores that data in the session, which is then saved in the user’s cookies. These cookies are also signed using the SECRET_KEY_BASE environment variable, which is required when deploying the application. This ensures that the cookies can be read but cannot be tampered with from the outside.

P.S. Do not be confused by signing_salt which is just a namespace for the signature.

Router module

Having our controller module, we are ready to set up routes. We’ll use the Phoenix Router, which provides built-in powerful macros for defining custom routes.

First, we will create endpoints for user sign-up and log-in and use our existing functions from the BackendWeb.AuthController module:

defmodule BackendWeb.Router do
  use BackendWeb, :router

  pipeline :auth do
    plug(:accepts, ["json"])
    plug(:fetch_session)
  end

  scope "/auth", BackendWeb do
    pipe_through(:auth)

    post("/login", AuthController, :login)
    post("/register", AuthController, :register)
  end
end

And by the way, analyze what is going on in this code:

  1. Via use BackendWeb, :router we import macros to define and handle requests while initializing the router in the current file.

  2. We create a pipeline of plugs and use two plugs which guarantees that they will block requests that do not accept JSON responses and fetch already existing sessions.

  3. We are specifying a namespace for our authentication routes, where each controller should be placed under BackendWeb module (on compilation it will prefix each controller module with this prefix).

  4. We define AuthController with login/2 and register/2 methods to handle users’ requests.

Tests

We will primarily write integration tests (through our controllers). In my opinion, there’s no need to add separate unit tests at this stage, as there isn’t much logic that a few well-written integration tests can’t cover. Additionally, Elixir has a great testing ecosystem and stack trace, so identifying the source of a bug in integration tests is generally straightforward.

To test our application we will use ExUnit and SeedFactory library. The last one is the library of my colleague Artur Plysyuk which makes testing isolated to the database toolkit (in Leyman terms you are free from fighting problems that are not related to your business logic).

To start writing tests with this library, you need to create a schema with commands that work with the testing context (literally any key-value object). This context will be responsible for maintaining the state of your application instead of a database.

See this example of a producing entity:

defmodule Backend.SeedFactorySchema do
  use SeedFactory.Schema

  command :create_user do
    param(:first_name, generate: &Faker.Person.first_name/0)
    param(:last_name, generate: &Faker.Person.last_name/0)
    param(:email, generate: &Faker.Internet.email/0)
    param(:preferred_currency, generate: &Faker.Currency.code/0)
    param(:password, generate: &Faker.String.base64/0)

    resolve(fn args ->
      with {:ok, user} <- Backend.Users.create_user(args) do
        {:ok, %{user: user}}
      end
    end)

    produce(:user)
  end
end

Here we define a command for user creation in a test context. On invocation, it will produce a user entity like this:

test "example", ctx
   ctx = ctx |> produce(:user)
 
   dbg(ctx) -->  #  ctx = %{
                 #    ... other fields from test context
                 #    user: %User{...}
                 #  }
end

It works in a way:

  1. The SeedFactory sorts commands in topological order to make sure that commands that rely on other entities are put at the back of the queue.

  2. If needed, the library creates all necessary dependencies for the first command in the queue (in our case there are no dependencies for the user).

  3. The library gets a resolver function from the first command in the queue and executes it.

  4. It checks the type of operation (create, update, or delete) and whether you do not fail constraints for such type of operation. For example, you are not allowed to recreate entities with the same entity name in one context. It’s done in this way because commands in the same context can rely on existing entities, and such manipulations will lead to unexpected errors.

  5. Saves the result of the operation to the context.

Having such an approach gives us the ability to work with our entities as plain objects and test them more elegantly.

Let’s also define a schema for the ability to create a test connection for authenticated and not authenticated users:

defmodule BackendWeb.SeedFactorySchema do
  use SeedFactory.Schema
  include_schema(Backend.SeedFactorySchema)

  command :build_conn do
    resolve(fn _ ->
      conn = Phoenix.ConnTest.build_conn()

      {:ok, %{conn: conn}}
    end)

    produce(:conn)
  end

  command :create_user_session do
    param(:user, entity: :user)
    param(:conn, entity: :conn, with_traits: [:unauthenticated])

    resolve(fn args ->
      conn = args.conn |> assign(:current_user, args.user) |> init_test_session(%{})

      {:ok, %{conn: conn}}
    end)

    update(:conn)
  end

  trait :unauthenticated, :conn do
    exec(:build_conn)
  end
  
  trait :user_session, :conn do
    from(:unauthenticated)
    exec(:create_user_session)
  end
end

At this point, some commands have dependencies — for example, :create_user_session requires :user entity. So the library will know that such a session requires the user, and will use such from the context (or create automatically if it does not exist).

One note: There’s also a concept called a trait, which you can think of as a modification to an existing entity. For example, an authenticated connection, after applying the :user_session trait becomes authenticated by a specific user.

Let’s write tests for sign-up with such an approach. We will test two cases:

  1. Register user with non-existing email (Should be allowed)

  2. Register user with existing email (Should be restricted) 

defmodule BackendWeb.Controllers.AuthTest do 
  use BackendWeb.ConnCase, async: true

  @api_auth_register_endpoint "/auth/register"

  test "register", ctx do
    ctx = ctx |> produce(conn: [:unauthenticated])

    email = Faker.Internet.email()
    password = Faker.String.base64()
    first_name = Faker.Person.first_name()
    last_name = Faker.Person.last_name()

    conn =
      post(ctx.conn, @api_auth_register_endpoint, %{
        "email" => email,
        "password" => password,
        "first_name" => first_name,
        "last_name" => last_name
      })

    assert %{
             "data" => %{
               "user" => %{
                  "email" => ^email,
                  "first_name" => ^first_name,
                  "last_name" => ^last_name
               }
             }
           } =
             Jason.decode!(conn.resp_body)
  end

  test "register with existing email", ctx do
    ctx = ctx |> produce([:user, conn: [:unauthenticated]])

    conn =
      post(ctx.conn, @api_auth_register_endpoint, %{
        "email" => ctx.user.email,
        "password" => ctx.user.password,
        "first_name" => Faker.Person.first_name(),
        "last_name" => Faker.Person.last_name()
      })

    assert_bad_request_response(conn, %{"email" => ["has already been taken"]})
  end
end

Does it look easier than it should? Yeah, I think so, too. I have been writing tests in such a way for several months now, and I’m really happy that there is a way to do so. Huge kudos to Artur Plysyuk!

Authentication middleware

The last thing that I want to do in this article is to show how to protect your routes only for authenticated users.

Let’s write such middleware:

defmodule BackendWeb.Plugs.EnsureAuthenticated do
  import Plug.Conn
  import BackendWeb.ResponseHelpers

  def init(opts), do: opts

  def call(%Plug.Conn{assigns: %{current_user: _current_user}} = conn, _opts), do: conn

  def call(conn, _opts) do
    current_user = get_session(conn, :current_user)

    if is_nil(current_user) do
      conn |> unauthorized_response() |> halt()
    else
      assign(conn, :current_user, current_user)
    end
  end
end

As you may remember, HTTP is stateless, so the user must be authorized on each request. This middleware checks whether the connection already has an authenticated user. If not, it attempts to retrieve the user from cookies that were previously set during the login or sign-up process. If that also fails, the middleware halts the connection and returns an authorization error. In this case, access to the resource will be denied.

We will put this middleware on our API to prevent unauthorized access like so:

defmodule BackendWeb.Router do
  use BackendWeb, :router
  
  pipeline :api do
    plug(:accepts, ["json"])
    plug(:fetch_session)
    plug BackendWeb.Plugs.EnsureAuthenticated
  end
  
  scope "/api", BackendWeb do
    pipe_through(:api)
    
    # ... our API routes
  end
  
  # other routes
end

P.S. Make sure you include the :fetch_session plug before any middleware that requires access to session data, such as cookies.

Conclusion

In this section, I’ve walked you through the complete cycle of developing an API feature, including authorization. I hope you enjoyed it, don’t forget to like and subscribe for further articles. I won’t repeat this process in the following sections, but I’ll refer you back to this article for similar development steps.