It's been a while since I last posted and for good reason...I've decided I need to attempt to compliment my backend skills with some frontend skills! I've never enjoyed writing javascript. Mostly because I've only ever used jQuery to added a little pizzazz to my sites(ie. toggleClass() and ajax()). I didn't have any javascript code organization. It was all spaghetti.

Javascript Revolution

Currently javascript is getting an insane amount of attention. New things are popping up all over the place(or maybe I've just been sheltered under my php rock). There are all sorts of frontend javascript frameworks now: Backbone, Angular, Ember, CanJS. Combine one of those with Twitter Boostrap or Zurb's Foundation and you can create some pretty slick frontends without doing much work.

Javascript is even becoming popular for the backend with the advent of Node. Node has spurred Express, Meteor, Sails, and more. I've got to say, programming both the backend and frontend with the same language is pretty darn appealing, even if I hate the fact that javascript isn't class based.

Where I'll start

Considering all that, I figure it would do my career well to really start giving javascript more of my attention. So where in the world do I start? Well, Ember seems to be the new hotness and even though I don't know a lick of ruby or rails, having a contributor to both rails and jQuery on the core team, gives me a warm fuzzy feeling. I'll start with Ember!

Ignition

After a couple of days of trying to create something I realized one thing: Ember is hard. Ember does a ton of magic for you behind the scenes. As a noob, this was a hindrance to me. I couldn't distinguish what Ember was doing for me and what it expected me to do. It was also extremely difficult to wrap my head around what MV* means in Ember. It doesn't resemble MVC in the backend at all. Once I realized and understood that, it all started to click and I've grown to really enjoy developing the frontend with Ember.

Another frustration of learning Ember/Ember-Data was the lack of up-to-date resources. I would constantly find stackoverflow questions that were "answered" and the answer referred to a jsfiddle that referenced "ember-latest.js" and guess what...Ember's api had changed, rendering it useless. So many times I found myself wanting to ditch it and stick to backend dev, but no, I don't want to be the guy unwilling to adapt. I. Must. Persevere.

I set out to build a tiny CRM application and implement some features I know I'll run into when developing something for real. Some features like:

• basic CRUD operations
• various relationship types, including a polymorphic relationship
• loading relationships asynchronously
• modals
• Select2 integration
• data normalization, serialization
• and more

I've recorded a 10 part video playlist of myself rebuilding the app so that you all may reap the benefits of my efforts. I know when I first set out to learn ember I got tired of being pointed to paid methods of learning the framework. This is me giving back for all of the help I've received on my quest. Enjoy.

Project Overview

Before we get started I think it makes the most sense to have a look at what we're going to build.

Part 1

Let's get to it. In this first episode we're going to cover:

• project structure
• libraries used
• model definitions
• route definitions

Part 2

Topics covered:

• creating Route objects
• retrieving models from the ember-data store/API
• expected JSON format
• displaying a list of models in a template
• using a few handlebars helpers

Part 3

Topics covered:

• creating a route that uses a dynamic segment
• using handlebars partials
• getting and displaying related models
• sending actions from templates to controllers
• using ember-data model flags
• rolling back model changes
• saving models

Part 4

Topics covered:

• reviewing relationship JSON
• creating a custom REST serializer

Part 5

Topics covered:

• creating a route for new models
• rendering a specific template from the route
• what JSON to return when saving models
• removing unused models
• cleaning up unsaved model changes

Part 6

Topics covered:

• hiding models based on properties
• creating mixins for our controllers
• handling backend errors
• using Twitter Bootstrap's alerts

Part 7

Topics covered:

• fixing a mixin bug
• reviewing my error response JSON

Part 8

Topics covered:

• implementing everything we've done for companies so far for people

Part 9

Topics covered:

• fixing a dynamic model property bug
• incorporating Select2 using a custom view
• using meta data in our JSON
• sending actions from a view

Part 10

Topics covered:

• using a Twitter Bootstrap modal
• rendering into named outlets
• using the Ember.Evented mixin
• sending actions from a view
• using two controllers in the same route
• binding attributes

<

p>

Questions, comments?

I hope you've found this useful. It was a really fun exercise and it sparked my interest to dive deeper into the javsacript world.

I'd love to hear any questions or comments. As I said in the first video, I want to learn from you guys as well. Please tell me if I'm doing something wrong or I could be doing something better!

I'm currently on a mission to learn ember.js. This is my first attempt at frontend development and ember seems to be the new hotness so I figured I'd give it a shot. I'm building the backend api with Laravel. Out of the box, Laravel is pretty well-suited for building an api. However, after learning what json responses ember expects, I realized there was some work to be done.

The dilemma

I was reading through the ember rest adapter guide and came across the model relationship expectations. Ember expects something like the following.

{
    "company": {
        "id": 1,
        "name": "Acme Company",
        "person_ids": [1, 2, 3]
    }
}

Notice how the relationship ids are contained in an array. Ember does support sideloading models like below.

{
    "company": {
        "id": 1,
        "name": "Acme Company",
        "person_ids": [1, 2, 3],
        "people": [{
            "id": 1,
            "name": "David Adams"
        },
        {
            "id": 2,
            "name": "John Doe"
        },
        {
            "id": 3,
            "name": "Bob Smith"
        }]
    }
}

The problem with this is that if your app is of any size, this isn't an option. Too much data would be loaded.

Laravel is awesome in that you can side/eager load models but there isn't an option for eager loading data outside of entire models. For example, the person_ids in our company models.

My first attempt

The very last section in the eloquent docs informs us that we can define custom accessors that don't have a corresponding column in the database. I thought I would use this feature to accomplish my goal.

class Company extends \Eloquent
{
    protected $table = 'companies';

    protected $appends = ['person_ids'];

    protected $person_ids;

    public function people()
    {
        return $this->hasMany('Person');
    }

    public function getPersonIdsAttribute()
    {
        if (is_null($this->person_ids)) {
            $ids = DB::table('people')->where('company_id', '=', $this->id)->lists('id');

            $this->person_ids = array_map('intval', $ids);
        }

        return $this->person_ids;
    }
}

This accomplishes my goal....but there is one big problem. This solution has the N + 1 query problem. I'm adding a lot of database overhead with this approach. Every model retrieved will require one more query to get all of its data. This isn't optimal at all.

The solution

Rather than require a separate query made by php, it would be better to have sql handle that for us behind the scenes. It would be great to be able to use a sql view to read the data from instead of directly off of the table.

I created a view with the following migration.

class CreateCompaniesView extends Migration 
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        DB::statement("CREATE VIEW companiesView AS
                        SELECT *,
                        (
                            SELECT GROUP_CONCAT(DISTINCT id SEPARATOR ',')
                            FROM people AS p
                            WHERE p.company_id = c.id
                        ) AS person_ids
                        FROM companies AS c");
    }

    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        DB::statement("DROP VIEW companiesView");
    }
}

This allows me to issue a query like SELECT * FROM companiesView and get the following resultset:

Now we need to figure out how to incorporate this view into our eloquent model.

Eloquent and views

We can't simply change the $table property on our model to the newly created view. Views cannot be written to, only read from. We need to provide the writable $table along with a $readFrom property where we can specify a view.

My first thought was to hook into the saving and saved eloquent model events to swap out the view for the real table when saving. I learned that the saving event isn't fired soon enough though. Eloquent has already built part of the query at that point.

Rather than using events, I decided to wrap eloquent with my own BaseModel class. This allows me to override the save() method and juggle the table/view before and after the actual save.

public function save(array $options = [])
{
    $this->toWriteMode();

    try {
        $saved = parent::save($options);
    } catch (\Exception $e) {
        $this->toReadMode();
        throw $e;
    }

    $this->toReadMode();

    return $saved;
}

The toWriteMode() and toReadMode() methods simply swap the $table to the appropriate table or view.

Read only attributes

The next problem is figuring out how to handle these read only attributes, ie. person_ids. These values need to be attributes on the models but not saved to the database.

We need a way to cache these read only values, remove them from being attributes on the model, save the model, and then add them back after the model is saved.

//part of the BaseModel class

    protected $readOnly = [];

    protected $readOnlyCache = [];

    public function save(array $options = [])
    {
        $this->toWriteMode();
        $this->cacheReadOnly();

        try {
            $saved = parent::save($options);
        } catch (\Exception $e) {
            $this->toReadMode();
            throw $e;
        }

        $this->toReadMode();
        $this->restoreReadOnly();

        return $saved;
    }

    protected function cacheReadOnly()
    {
        $this->readOnlyCache = [];

        foreach ($this->readOnly as $key) {
            $value = $this->getAttributeValue($key);
            $this->readOnlyCache[$key] = $value;
            $this->__unset($key);
        }
    }

    protected function restoreReadOnly()
    {
        foreach ($this->readOnlyCache as $key => $value) {
            $this->setAttribute($key, $value);
        }
    }

Sweet. Now in our model classes, we just need to provide an array of $readOnly attributes.

Putting it all together

With the creation of this base model class we can define a company model like below.

class Company extends BaseModel
{
    protected $table = 'companies';

    protected $fillable = ['name'];

    protected $guarded = ['id'];

    public function people()
    {
        return $this->hasMany('Person');
    }
} 

There's absolutely nothing new in this Company class. It works just as you would expect. However, now we can extend this class to create an EagerCompany class.

class EagerCompany extends Company
{
    protected $readFrom = 'companiesView';

    protected $readOnly = ['person_ids'];

    public function getPersonIdsAttribute($ids)
    {
        return $this->intArrayAttribute($ids);
    }
} 

This class will read its data from the view we created earlier and we can save and retrieve it as normal. It will fetch our read only attributes and they will be handled appropriately when saving.

That new intArrayAttribute() method just converts the comma delimited id string returned from the view into an array of integers.

We have some options now. We can use the Company internally but if we need those extra read only attributes, say in an api response, we can use the EagerCompany class.

I found this to work quite well for getting a list of ids for relationships. I also plan to use this same idea for returning counts of relationships.

{
    "company": {
        "id": 1,
        "name": "Acme Company",
        "person_ids": [1, 2, 3],
        "people_count": 3
    }
}

The base model

I've only shown you bits and pieces of this base model up until now. Here's the whole enchilada.

/**
 * Base model for allowing reads from sql views
 *
 * Class BaseModel
 * @package ProgrammingAreHard\Models
 */
abstract class BaseModel extends \Eloquent
{
    /**
     * Writable table
     *
     * @var string
     */
    protected $writeTable;

    /**
     * Read table/view
     *
     * @var string
     */
    protected $readFrom;

    /**
     * Read only attributes (not to be saved)
     *
     * @var array
     */
    protected $readOnly = [];

    /**
     * Cache for read only attribute values
     *
     * @var array
     */
    protected $readOnlyCache = [];

    /**
     * Instantiate and set the table
     *
     * @param array $attributes
     */
    public function __construct(array $attributes = [])
    {
        parent::__construct($attributes);

        $this->writeTable = $this->getTable();

        $this->toReadMode();
    }

    /**
     * Juggle the table and read only attributes
     *
     * @param array $options
     * @return bool
     * @throws \Exception
     */
    public function save(array $options = [])
    {
        $this->toWriteMode();
        $this->cacheReadOnly();

        try {
            $saved = parent::save($options);
        } catch (\Exception $e) {
            $this->toReadMode();
            throw $e;
        }

        $this->toReadMode();
        $this->restoreReadOnly();

        return $saved;
    }

    /**
     * Cache and remove read only attributes
     *
     * @return void
     */
    protected function cacheReadOnly()
    {
        $this->readOnlyCache = [];

        foreach ($this->readOnly as $key) {
            $value = $this->getAttributeValue($key);
            $this->readOnlyCache[$key] = $value;
            $this->__unset($key);
        }
    }

    /**
     * Restore the cached read only attributes
     *
     * @return void
     */
    protected function restoreReadOnly()
    {
        foreach ($this->readOnlyCache as $key => $value) {
            $this->setAttribute($key, $value);
        }
    }

    /**
     * Get the writable table
     *
     * @return string
     */
    public function getWriteTable()
    {
        return $this->writeTable;
    }

    /**
     * Swap to the writable table
     *
     * @return void
     */
    protected function toWriteMode()
    {
        $this->setTable($this->getWriteTable());
    }

    /**
     * Get the readable table/view
     *
     * @return string
     * @throws \Exception
     */
    public function getReadFrom()
    {
        if (is_null($this->readFrom)) {
            $this->readFrom = $this->getWriteTable();
        }

        return $this->readFrom;
    }

    /**
     * Swap to the readable table/view
     *
     * @return void
     */
    protected function toReadMode()
    {
        $this->setTable($this->getReadFrom());
    }

    /**
     * Convert the value to an int array if needed
     *
     * @param string|array $value
     * @return array
     */
    protected function intArrayAttribute($value)
    {
        if (is_string($value) and strlen($value) > 0) {
            $value = explode(',', $value);
            $value = array_map('intval', $value);
        }

        return is_array($value) ? $value : [];
    }
}

The controller

Here's a very simplified version of my controller.

class CompaniesController extends \BaseController 
{
    /**
    * Display a listing of companies.
    *
    * @return Response
    */
    public function index()
    {
        $res = new stdClass;
        $res->companies = EagerCompany::all()->toArray();

        return Response::json($res);
    }

    /**
    * Display a single company.
    *
    * @param  int  $id
    * @return Response
    */
    public function show($id)
    {
        $res = new stdClass;
        $res->company = EagerCompany::find($id)->toArray();

        return Response::json($res);
    }
}

I feel like this is a pretty good way to handle eager loading of non-model data. Maybe there's a better way to do this. I'd love to hear any ideas if you have some!

Abstraction layers in your projects allow for extreme flexibility. That's a given. Define a QueueInterface and create implementations for Beanstalkd, Amazon SQS, and IronMQ. Swap them out without touching anything but your IoC container. That's awesome.

I was recently introduced to the repository pattern, a type of abstraction and organizational technique. The idea being, create a repository for each of your models to retrieve and persist to and from. A supposed benefit of the repository pattern is the ability to abstract your ORM and create different implemenations for Eloquent, Doctrine, Propel, etc. This abstraction intrigued me. I set off to put this idea into practice and see what it took. Here are my findings.

The goal

My goal for this project was to abstract my ORM. Pretty straightforward, right? Just take the repository pattern to the extreme. Way easier said than done.

Time to go to work

I decided my first task was to create a UserRepository interface. An interface that had method stubs like create(array $fieldValues), findById($id), and currentUser(), etc.

Next up was to create my first implementation. Because Laravel is my framework, it made sense to implement the Eloquent ORM first. That was pretty easy, but then I realized something...my EloquentUserRepository is going to return Eloquent models. Okay...need to create a UserInterface.

Looks like I need to abstract Eloquent's sexy dynamic setter/getter action. I created my UserInterface with setEmail($email), getEmail(), setPassword($password), and getPassword(), save() methods.

Persisting for ORM X

At this point I started thinking I should peek at some other ORMs to see how they do things. Doctrine was in my line of sights. As soon as I learned about Doctrine's EntityManager I knew architecting this thing was going to be difficult.

In Doctrine, you wouldn't typically call save() on your models. You use the EntityManager to persist($model) and flush() the models. This wraps the save in a SQL transaction for you. I like that a lot. It means transactions are handled for you when you do something like the following.

$user = new User();
$user->setName('David');

$profile = new UserProfile();
$profile->setLocation('Dallas');
$profile->setOccupation('Software Developer');
// relate this profile to the user
$profile->setUser($user);

$em = $this->getDoctrine()->getManager();
$em->persist($user);
$em->persist($profile);
$em->flush();

I want to use transactions with Eloquent, so it looks like I need to add a save(UserInterface $user) method to the UserRepository interface. I'll just call save() on the Eloquent model inside of the EloquentUserRepository's save(UserInterface $user) method.

Hmm...what about relationships?

After looking at Doctrine, I realized I needed to create different method stubs somewhere for relating and un-relating models.

I wonder how Propel handles relationships. Hmm...looks like you set related models on each other and then save() it, which cascades the saves.

//Propel relationship saving
$author = new Author();
$author->setFirstName("Leo");
$author->setLastName("Tolstoy");
// no need to save the author yet

$publisher = new Publisher();
$publisher->setName("Viking Press");
// no need to save the publisher yet

$book = new Book();
$book->setTitle("War & Peace");
$book->setIsbn("0140444173");
$book->setPublisher($publisher);
$book->setAuthor($author);
$book->save(); // saves all 3 objects!

About that...do I create a UserRelationship interface? Do I add methods to the UserInterface? How should I go about handling the cascading saves by Propel versus the immediate saves done by Eloquent versus the flushing done by Doctrine? Uhhh...I'll just not worry about that for now. Validation is what I need next!

Validation

I like the idea of validating data for models before I stick the data inside the model. In Laravel, I can just create a create a class specifically for validation and use Laravel's validation class. I'll do that and validate an array of POST data from the request.

Just to be safe, I better take a peek at how Doctrine handles validation. Umm...a bit different. Looks like I can get a handle on a validator object and pass it a model or I can create a some kind of form object from my models and validate it by passing it a Request object. Looks like a need to create interfaces for each of my form validators and have different implementations for each ORM. That sucks.

Wonder how Propel handles validation. Looks like you validate a Propel model by calling a validate() on the model itself. Great...so Laravel validates an array of data, Doctrine a Request object, and Propel the models themselves. How do I interface that?

Observers

I think I want to be able to hook into different events and have event listeners for certain actions. Eloquent already has some Eloquent events defined for saving, updating, deleting models. That's nice but no good for me. I need generic events being fired.

I need to fire those same events in all of my UserRepository implementations. Yet another thing to keep in mind.

This is rough

Geez, I had no idea how much planning this would take. I'm not getting anywhere. Although a cool idea, this proof of concept isn't worth the time I'm spending on it.

What do you have to show for this?

Not much! :( I wish I had some cool demo where I could change a flag in the code and Eloquent would be used, change it again and Doctrine would be used, change it again and Propel would be used...but, alas, I do not.

The truth is I've spent so much time reading about how these different ORM's handle different aspects and not writing any code. After learning how different each ORM is, I've found it incredibly difficult to interface the different components to make an ORM agnostic project work.

I do not want to sound like I'm whining about something I was promised by using the repository pattern. That is not the intention of this post at all. The repository pattern most definitely has its benefits, but completely abstracting your ORM was a little too difficult for me. It was hindering me from actually producing anything.

It's not all doom and gloom for me though. This forced me to take a look at Doctrine(which I'm liking the more I see) and Propel. Learning how these different ORM's handle validation, relationships, transacations, etc has been a great experience for me. It has helped me form some opinions about what I like and don't like in an ORM, which is great. I didn't even think about that indirect benefit when I started.

As of right now, for me, I've decided that I can use the repository pattern for organizational purposes, but not ORM abstraction. I think I need to pick an ORM and stick with it to get anything done.

Overall I'm really glad I attempted this. I learned a lot. I hope this post starts a discussion on proper ORM abstraction. I would love to here your guys' thoughts on the subject.