Refactor base and sql connector

.jshintrc

@@ -1,23 +1,23 @@
 {
-"node": true,
+  "node": true,
-"browser": true,
+  "browser": true,
-"camelcase" : true,
+  "camelcase": true,
-"eqnull" : true,
+  "eqnull": true,
-"indent": 2,
+  "indent": 2,
-"undef": true,
+  "undef": true,
-"unused": true,
+  "unused": "vars",
-"quotmark": "single",
+  "quotmark": "true",
-"maxlen": 90,
+  "maxlen": 110,
-"trailing": true,
+  "trailing": true,
-"newcap": true,
+  "newcap": true,
-"nonew": true,
+  "nonew": true,
-"sub": true,
+  "sub": true,
-"globals": {
+  "globals": {
-  "describe": true,
+    "describe": true,
-  "it": true,
+    "it": true,
-  "before": true,
+    "before": true,
-  "beforeEach": true,
+    "beforeEach": true,
-  "after": true,
+    "after": true,
-  "afterEach": true
+    "afterEach": true
-}
+  }
 }

docs.json

@@ -0,0 +1,22 @@
+{
+  "content": [
+    {
+      "title": "Build a SQL connector",
+      "depth": 2
+    },
+    "docs/sql-connector.md",
+    {
+      "title": "Base Connector",
+      "depth": 2
+    },
+    "lib/connector.js",
+    {
+      "title": "SQL Connector",
+      "depth": 2
+    },
+    "lib/sql.js",
+    "lib/parameterized-sql.js"
+  ],
+  "codeSectionDepth": 3
+}
+

docs/sql-connector.md

@@ -0,0 +1,662 @@
+# Build a connector for relational databases
+
+This tutorial walks you through the MySQL connector implementation to teach you
+how to develop a connector for relational databases.
+
+## Understand a connector's responsibilities
+
+In LoopBack, models encapsulate business data and logic as JavaScript properties
+and methods. One of the powerful features of LoopBack is that application 
+developers don't have to implement all behaviors for their models as a lot of 
+them are already provided by the framework out of box with data sources and 
+connectors. For example, a model automatically receives the create, retrieve, 
+update, and delete (CRUD) functions if it is attached to a data source for a
+database. LoopBack abstracts the persistence layer and other backend services, 
+such as REST APIs, SOAP web services, and storage services, and so on, as data 
+sources, which are configurations of backend connectivity and integration. 
+Each data source is backed a connector which implements the interactions between
+Node.js and its underlying backend system. Connectors are responsible for 
+mapping model method invocations to backend functions, such as database 
+operations or call to REST or SOAP APIs. The following diagram illustrates how
+connectors fit into the big picture of LoopBack API framework.
+ 
+![connector-architecture](connector-architecture.png)
+
+Please note that you don't always have to develop a connector to allow your 
+application to interact with other systems. Ad-hoc integration can be done with 
+custom methods on the model. The custom methods can be implemented using other 
+Node modules, such as drivers or clients to your backend.
+
+You should consider to develop a connector for common and reusable backend 
+integrations, for example:
+
+- Integrate with a backend such as databases
+- Reusable logic to interact with another system
+
+There are a few typical types of connectors based on what backends they connect
+to and interact with. 
+
+- Databases that support full CRUD operations
+  - Oracle, SQL Server, MySQL, Postgresql, MongoDB, In-memory DB
+- Other forms of existing APIs
+  - REST APIs exposed by your backend
+  - SOAP/HTTP web services
+- Services
+  - E-mail
+  - Push notification
+  - Storage
+
+The connectors are mostly transparent to models. Their functions are mixed into
+model classes through data source attachments.
+
+Most of the connectors need to implement the following logic:
+
+- Lifecycle handlers
+  - initialize: receive configuration from the data source settings and 
+    initialize the connector instance 
+  - connect: create connections to the backend system
+  - disconnect: close connections to the backend system
+  - ping (optional): check if the connectivity 
+  
+- Model method delegations
+  - Delegating model method invocations to backend calls, for example CRUD
+  
+- Connector metadata (optional)
+  - Model definition for the configuration, such as host/url/user/password
+  - What data access interfaces are implemented by the connector (the capability 
+    of the connector)
+  - Connector-specific model/property mappings
+
+To mixin methods onto model classes, a connector must choose what functions to
+offer. Different types of connectors implement different interfaces that group
+a set of common methods, for example:
+
+- Database connectors
+  - CRUD methods, such as create, find, findById, deleteAll, updateAll, count
+  
+- E-mail connector
+  - send()
+
+- Storage connector
+  - Container/File operations, such as createContainer, getContainers, getFiles,
+    upload, download, deleteFile, deleteContainer
+    
+- Push Notification connector
+  - notify()
+
+- REST connector
+  - Map operations from existing REST APIs
+  
+- SOAP connector
+  - Map WSDL operations
+
+In this tutorial, we'll focus on building a connector for databases that provide
+the full CRUD capabilities.
+
+## Understand a database connector with CRUD operations 
+
+![crud-connector](crud-connector.png)
+
+LoopBack unifies all CRUD based database connectors so that a model can choose
+to attach to any of the supported database. There are a few classes involved 
+here:
+
+1. [PersistedModelClass](http://docs.strongloop.com/display/public/LB/PersistedModel+class) 
+defines all the methods mixed into a model for persistence.
+
+2. [The DAO facade](https://github.com/strongloop/loopback-datasource-juggler/blob/master/lib/dao.js) 
+maps the PersistedModel methods to connector implementations.
+
+3. CRUD methods need to be implemented by connectors
+
+
+
+In the next sections, we will use MySQL connector as an example to walk through
+how to implement a SQL based connector.
+
+## Define a module and export the *initialize* function
+
+A LoopBack connector is packaged as a Node.js module that can be installed using
+`npm install`. LoopBack runtime loads the module via `require` on behalf of 
+data source configuration, for example, `require('loopback-connector-mysql');`. 
+The connector module should export an `initialize` function as follows: 
+
+```js
+// Require the DB driver
+var mysql = require('mysql'); 
+// Require the base SqlConnector class
+var SqlConnector = require('loopback-connector').SqlConnector;
+// Require the debug module with a pattern of loopback:connector:connectorName
+var debug = require('debug')('loopback:connector:mysql');
+
+/**
+ * Initialize the MySQL connector against the given data source
+ *
+ * @param {DataSource} dataSource The loopback-datasource-juggler dataSource
+ * @param {Function} [callback] The callback function
+ */
+exports.initialize = function initializeDataSource(dataSource, callback) {
+  ...
+};
+```
+
+After the initialization, the dataSource object will have the following properties 
+added:
+
+- connector: The connector instance
+- driver: The module for the underlying database driver (`mysql` for MySQL)
+
+The `initialize` function should calls the `callback` function once the connector
+has been initialized. 
+
+## Create a subclass of SqlConnector
+
+Connectors for relational databases have a lot of things in common. They are 
+responsible for mapping CRUD operations to SQL statements. LoopBack provides a
+base class called `SqlConnector` that encapsulates the common logic for inheritance.
+The following code snippet is used to create a subclass of SqlConnector.
+
+```js
+/**
+ * @constructor
+ * Constructor for MySQL connector
+ * @param {Object} settings The data source settings
+ */
+function MySQL(settings) {
+  // Call the super constructor with name and settings
+  SqlConnector.call(this, 'mysql', settings);
+  ...
+}
+// Set up the prototype inheritence
+require('util').inherits(MySQL, SqlConnector);
+```
+
+## Implement methods to interact with the database
+
+A connector implements the following methods to communicate with the underlying
+database.
+
+### Connect to the database
+
+The `connect` method establishes connections to the database. In most cases, a
+connection pool will be created based on the data source settings, including
+`host`, `port`, `database`, and other configuration properties.
+
+```js
+MySQL.prototype.connect = function (cb) {
+  // ...
+};
+```
+
+### Disconnect from the database
+
+The `disconnect` method close connections to the database. Most database drivers
+provide APIs.
+
+```js
+/**
+ * Disconnect from MySQL
+ */
+MySQL.prototype.disconnect = function (cb) {
+  // ...
+};
+```   
+### Ping the database
+
+The `ping` method tests if the connection to the database is healthy. Most
+connectors choose to implement it by executing a simple SQL statement.
+
+```js   
+MySQL.prototype.ping = function(cb) {
+  // ...
+};
+```
+
+## Implement CRUD methods
+
+The connector is responsible for implementing the following CRUD methods. The 
+good news is that the base SqlConnector now have most of the methods implemented
+with the extension point to override certain behaviors that are specific to the
+underlying database.
+
+To extend from SqlConnector, the minimum set of methods below must be 
+implemented:
+
+
+### Execute a SQL statement with parameters
+
+The `executeSQL` method is the core function that a connector has to implement. 
+Most of other CRUD methods are delegated to the `query` function. It executes 
+a SQL statement with an array of parameters. `SELECT` statements will produce 
+an array of records representing matching rows from the database while other 
+statements such as `INSERT`, `DELETE`, or `UPDATE` will report the number of 
+rows changed during the operation.
+
+```js
+/**
+ * Execute the parameterized sql statement
+ *
+ * @param {String} sql The SQL statement, possibly with placeholders for parameters
+ * @param {*[]} [params] An array of parameter values
+ * @param {Objet} [options] Options passed to the CRUD method
+ * @param {Function} [callback] The callback after the SQL statement is executed
+ */
+MySQL.prototype.executeSQL = function (sql, params, options, callback) {
+  // ...
+};
+```
+
+### Map values between a model property and a database column
+
+```js
+/**
+ * Converts a model property value into the form required by the
+ * database column. The result should be one of following forms:
+ *
+ * - {sql: "point(?,?)", params:[10,20]}
+ * - {sql: "'John'", params: []}
+ * - "John"
+ *
+ * @param {Object} propertyDef Model property definition
+ * @param {*} value Model property value
+ * @returns {ParameterizedSQL|*} Database column value.
+ *
+ */
+SqlConnector.prototype.toColumnValue = function(propertyDef, value) {
+  /*jshint unused:false */
+  throw new Error('toColumnValue() must be implemented by the connector');
+};
+
+/**
+ * Convert the data from database column to model property
+ * @param {object} propertyDef Model property definition
+ * @param {*) value Column value
+ * @returns {*} Model property value
+ */
+SqlConnector.prototype.fromColumnValue = function(propertyDef, value) {
+  /*jshint unused:false */
+  throw new Error('fromColumnValue() must be implemented by the connector');
+};
+```
+
+### Helpers to generate SQL statements and parse responses from DB drivers
+
+```js
+/**
+ * Build a new SQL statement with pagination support by wrapping the given sql
+ * @param {String} model The model name
+ * @param {ParameterizedSQL} stmt The sql statement
+ * @param {Number} limit The maximum number of records to be fetched
+ * @param {Number} offset The offset to start fetching records
+ * @param {String[]} order The sorting criteria
+ */
+SqlConnector.prototype.applyPagination = function(model, stmt, filter) {
+  /*jshint unused:false */
+  throw new Error('applyPagination() must be implemented by the connector');
+};
+
+/**
+ * Parse the result for SQL UPDATE/DELETE/INSERT for the number of rows
+ * affected
+ * @param {String} model Model name
+ * @param {Object} info Status object
+ * @returns {Number} Number of rows affected
+ */
+SqlConnector.prototype.getCountForAffectedRows = function(model, info) {
+  /*jshint unused:false */
+  throw new Error('getCountForAffectedRows() must be implemented by the connector');
+};
+
+/**
+ * Parse the result for SQL INSERT for newly inserted id
+ * @param {String} model Model name
+ * @param {Object} info The status object from driver
+ * @returns {*} The inserted id value
+ */
+SqlConnector.prototype.getInsertedId = function(model, info) {
+  /*jshint unused:false */
+  throw new Error('getInsertedId() must be implemented by the connector');
+};
+
+/**
+ * Escape the name for the underlying database
+ * @param {String} name The name
+ * @returns {String} An escaped name for SQL
+ */
+SqlConnector.prototype.escapeName = function(name) {
+  /*jshint unused:false */
+  throw new Error('escapeName() must be implemented by the connector');
+};
+
+/**
+ * Escape the name for the underlying database
+ * @param {String} value The value to be escaped
+ * @returns {*} An escaped value for SQL
+ */
+SqlConnector.prototype.escapeValue = function(value) {
+  /*jshint unused:false */
+  throw new Error('escapeValue() must be implemented by the connector');
+};
+
+/**
+ * Get the place holder in SQL for identifiers, such as ??
+ * @param {String} key Optional key, such as 1 or id
+ * @returns {String} The place holder
+ */
+SqlConnector.prototype.getPlaceholderForIdentifier = function(key) {
+  /*jshint unused:false */
+  throw new Error('getPlaceholderForIdentifier() must be implemented by the connector');
+};
+
+/**
+ * Get the place holder in SQL for values, such as :1 or ?
+ * @param {String} key Optional key, such as 1 or id
+ * @returns {String} The place holder
+ */
+SqlConnector.prototype.getPlaceholderForValue = function(key) {
+  /*jshint unused:false */
+  throw new Error('getPlaceholderForValue() must be implemented by the connector');
+};
+```
+
+### Override other methods
+
+There are a list of methods that serve as default implementations in the SqlConnector.
+The connector can choose to override such methods to customize the behaviors. Please
+see a complete list at http://apidocs.strongloop.com/loopback-connector/.
+
+## Implement database/model synchronization methods
+
+It's often desirable to apply model definitions to the underlying relational
+database to provision or update schema objects so that they stay synchronized
+with the model definitions.
+
+### automigrate and autoupdate 
+
+There are two flavors:
+
+- automigrate - Drop existing schema objects if exist and create them based on 
+model definitions. Existing data will be lost.
+- autoupdate - Detects the difference between schema objects and model 
+definitions, alters the database schema objects. Existing data will be kept.
+
+```js
+/**
+ * Perform autoupdate for the given models
+ * @param {String[]} [models] A model name or an array of model names.
+ * If not present, apply to all models
+ * @param {Function} [cb] The callback function
+ */
+MySQL.prototype.autoupdate = function (models, cb) {
+  // ...
+};
+
+MySQL.prototype.automigrate = function (models, cb) {
+  // ...
+};
+```
+
+The `automigrate` and `autoupdate` operations are usually mapped to a sequence of 
+DDL statements.
+
+### Build a CREATE TABLE statement
+
+```js
+/**
+ * Create a DB table for the given model
+ * @param {string} model Model name
+ * @param cb
+ */
+MySQL.prototype.createTable = function (model, cb) {
+  // ...
+};
+```
+
+### Check if models have corresponding tables
+
+```js
+/**
+ * Check if the models exist
+ * @param {String[]} [models] A model name or an array of model names. If not
+ * present, apply to all models
+ * @param {Function} [cb] The callback function
+ */
+MySQL.prototype.isActual = function(models, cb) {
+  // ...
+};
+```
+
+### Alter a table 
+
+```js
+MySQL.prototype.alterTable = function (model, actualFields, actualIndexes, done, checkOnly) {
+  // ...
+};
+```
+
+### Build column definition clause for a given model
+
+```js
+MySQL.prototype.buildColumnDefinitions =
+MySQL.prototype.propertiesSQL = function (model) {
+  // ...
+};
+```
+
+### Build index definition clause for a given model property
+
+```js
+MySQL.prototype.buildIndex = function(model, property) {
+  // ...
+};
+```
+
+### Build indexes for a given model
+
+```js
+MySQL.prototype.buildIndexes = function(model) {
+  // ...
+};
+```
+
+### Build column definition for a given model property
+
+```js
+MySQL.prototype.buildColumnDefinition = function(model, prop) {
+  // ...
+};
+```
+
+### Build column type for a given model property
+
+```js
+MySQL.prototype.columnDataType = function (model, property) {
+  // ...
+};
+```
+
+## Implement model discovery from database schemas
+
+For relational databases that have schema definitions, the connector can 
+implement the discovery capability to reverse engineer database schemas into
+model definitions.
+
+### Build a SQL statement to list schemas
+
+```js
+/**
+ * Build sql for listing schemas (databases in MySQL)
+ * @params {Object} [options] Options object
+ * @returns {String} The SQL statement
+ */
+ function querySchemas(options) {
+   // ...
+ }
+```  
+
+### Build a SQL statement to list tables
+
+```js
+/**
+ * Build sql for listing tables
+ * @param options {all: for all owners, owner|schema: for a given owner}
+ * @returns {string} The sql statement
+ */
+ function queryTables(options) {
+   // ...
+ }
+```js
+  
+### Build a SQL statement to list views  
+
+```js
+/**
+ * Build sql for listing views
+ * @param options {all: for all owners, owner: for a given owner}
+ * @returns {string} The sql statement
+ */
+ function queryViews(options) {
+   // ...
+ }
+```  
+  
+### Discover schemas  
+
+```js
+ MySQL.prototype.discoverDatabaseSchemas = function(options, cb) {
+   // ...
+ };
+```
+  
+### Discover a list of models  
+
+```js
+/**
+ * Discover model definitions
+ *
+ * @param {Object} options Options for discovery
+ * @param {Function} [cb] The callback function
+ */
+ MySQL.prototype.discoverModelDefinitions = function(options, cb) {
+   // ...
+ };
+```
+  
+### Discover a list of model properties for a given table
+   
+```js
+/**
+ * Discover model properties from a table
+ * @param {String} table The table name
+ * @param {Object} options The options for discovery
+ * @param {Function} [cb] The callback function
+ *
+ */
+ MySQL.prototype.discoverModelProperties = function(table, options, cb) {
+   // ...
+ };
+```
+
+### Discover primary keys for a given table
+
+```js
+/**
+ * Discover primary keys for a given table
+ * @param {String} table The table name
+ * @param {Object} options The options for discovery
+ * @param {Function} [cb] The callback function
+ */
+ MySQL.prototype.discoverPrimaryKeys = function(table, options, cb) {
+   // ...
+ };
+``` 
+
+### Discover foreign keys for a given table
+
+```js
+/**
+ * Discover foreign keys for a given table
+ * @param {String} table The table name
+ * @param {Object} options The options for discovery
+ * @param {Function} [cb] The callback function
+ */
+ MySQL.prototype.discoverForeignKeys = function(table, options, cb) {
+   // ...
+ };
+```
+
+### Discover exported foreign keys for a given table
+
+```js
+/**
+ * Discover foreign keys that reference to the primary key of this table
+ * @param {String} table The table name
+ * @param {Object} options The options for discovery
+ * @param {Function} [cb] The callback function
+ */
+ MySQL.prototype.discoverExportedForeignKeys = function(table, options, cb) {
+   // ...
+ };
+ ```
+ 
+### Discover indexes for a given table
+ 
+```js
+  MySQL.prototype.discoverIndexes = function(table, options, cb) {
+    // ...
+  };
+```  
+
+### Map column definition to model property definition
+
+```js
+  MySQL.prototype.buildPropertyType = function(columnDefinition) {
+    // ...
+  }
+``` 
+
+### Build SQL statements to discover database objects
+
+```js
+/**
+ * Build the sql statement to query columns for a given table
+ * @param schema
+ * @param table
+ * @returns {String} The sql statement
+ */
+ function queryColumns(schema, table) {
+   // ...
+ }
+ 
+/**
+ * Build the sql statement for querying primary keys of a given table
+ * @param schema
+ * @param table
+ * @returns {string}
+ */
+ function queryPrimaryKeys(schema, table) {
+   // ...
+ } 
+ 
+/**
+ * Build the sql statement for querying foreign keys of a given table
+ * @param schema
+ * @param table
+ * @returns {string}
+ */
+ function queryForeignKeys(schema, table) {
+   // ...
+ }
+ 
+/**
+ * Retrieves a description of the foreign key columns that reference the
+ * given table's primary key columns (the foreign keys exported by a table).
+ * They are ordered by fkTableOwner, fkTableName, and keySeq.
+ * @param schema
+ * @param table
+ * @returns {string}
+ */
+ function queryExportedForeignKeys(schema, table) {
+   // ...
+ } 
+```    

index.js

@@ -1,2 +1,4 @@
 exports.Connector = require('./lib/connector');
-exports.SqlConnector = require('./lib/sql');
+// Set up SqlConnector as an alias to SQLConnector
+exports.SQLConnector = exports.SqlConnector = require('./lib/sql');
+exports.ParameterizedSQL = exports.SQLConnector.ParameterizedSQL;

lib/connector.js

@@ -1,7 +1,9 @@
+var debug = require('debug')('loopback:connector');
+
 module.exports = Connector;
 
 /**
- * Base class for LooopBack connector. This is more a collection of useful
+ * Base class for LoopBack connector. This is more a collection of useful
  * methods for connectors than a super class
  * @constructor
  */
@@ -17,6 +19,15 @@ function Connector(name, settings) {
  */
 Connector.prototype.relational = false;
 
+/**
+ * Check if the connector is for a relational DB
+ * @returns {Boolean} true for relational DB
+ */
+Connector.prototype.isRelational = function() {
+  return this.isRelational ||
+    (this.getTypes().indexOf('rdbms') !== -1);
+};
+
 /**
  * Get types associated with the connector
  * @returns {String[]} The types for the connector
@@ -27,9 +38,11 @@ Connector.prototype.getTypes = function() {
 
 /**
  * Get the default data type for ID
+ * @param prop Property definition
  * @returns {Function} The default type for ID
  */
-Connector.prototype.getDefaultIdType = function() {
+Connector.prototype.getDefaultIdType = function(prop) {
+  /*jshint unused:false */
   return String;
 };
 
@@ -38,15 +51,16 @@ Connector.prototype.getDefaultIdType = function() {
  * @returns {Object} The metadata object
  * @property {String} type The type for the backend
  * @property {Function} defaultIdType The default id type
- * @property {Boolean} [isRelational] If the connector represents a relational database
+ * @property {Boolean} [isRelational] If the connector represents a relational
+ * database
  * @property {Object} schemaForSettings The schema for settings object
  */
-Connector.prototype.getMedadata = function () {
+Connector.prototype.getMetadata = function() {
   if (!this._metadata) {
     this._metadata = {
       types: this.getTypes(),
       defaultIdType: this.getDefaultIdType(),
-      isRelational: this.isRelational || (this.getTypes().indexOf('rdbms') !== -1),
+      isRelational: this.isRelational(),
       schemaForSettings: {}
     };
   }
@@ -55,13 +69,51 @@ Connector.prototype.getMedadata = function () {
 
 /**
  * Execute a command with given parameters
- * @param {String} command The command such as SQL
+ * @param {String|Object} command The command such as SQL
- * @param {Object[]} [params] An array of parameters
+ * @param {*[]} [params] An array of parameter values
+ * @param {Object} [options] Options object
  * @param {Function} [callback] The callback function
  */
-Connector.prototype.execute = function (command, params, callback) {
+Connector.prototype.execute = function(command, params, options, callback) {
-  /*jshint unused:false */
+  throw new Error('execute() must be implemented by the connector');
-  throw new Error('query method should be declared in connector');
+};
+
+/**
+ * Get the model definition by name
+ * @param {String} modelName The model name
+ * @returns {ModelDefinition} The model definition
+ */
+Connector.prototype.getModelDefinition = function(modelName) {
+  return this._models[modelName];
+};
+
+/**
+ * Get connector specific settings for a given model, for example,
+ * ```
+ * {
+ *   "postgresql": {
+ *     "schema": "xyz"
+ *   }
+ * }
+ * ```
+ *
+ * @param {String} modelName Model name
+ * @returns {Object} The connector specific settings
+ */
+Connector.prototype.getConnectorSpecificSettings = function(modelName) {
+  var settings = this.getModelDefinition(modelName).settings || {};
+  return settings[this.name];
+};
+
+/**
+ * Get model property definition
+ * @param {String} modelName Model name
+ * @param {String} propName Property name
+ * @returns {Object} Property definition
+ */
+Connector.prototype.getPropertyDefinition = function(modelName, propName) {
+  var model = this.getModelDefinition(modelName);
+  return model && model.properties[propName];
 };
 
 /**
@@ -69,10 +121,10 @@ Connector.prototype.execute = function (command, params, callback) {
  * @param {String} model The model name
  * @returns {DataSource} The data source
  */
-Connector.prototype.getDataSource = function (model) {
+Connector.prototype.getDataSource = function(model) {
-  var m = this._models[model];
+  var m = this.getModelDefinition(model);
   if (!m) {
-    console.trace('Model not found: ' + model);
+    debug('Model not found: ' + model);
   }
   return m && m.model.dataSource;
 };
@@ -82,7 +134,7 @@ Connector.prototype.getDataSource = function (model) {
  * @param {String} model The model name
  * @returns {String} The id property name
  */
-Connector.prototype.idName = function (model) {
+Connector.prototype.idName = function(model) {
   return this.getDataSource(model).idName(model);
 };
 
@@ -91,7 +143,7 @@ Connector.prototype.idName = function (model) {
  * @param {String} model The model name
  * @returns {[String]} The id property names
  */
-Connector.prototype.idNames = function (model) {
+Connector.prototype.idNames = function(model) {
   return this.getDataSource(model).idNames(model);
 };
 
@@ -102,8 +154,8 @@ Connector.prototype.idNames = function (model) {
  * @returns {Number} The id index, undefined if the property is not part
  *   of the primary key
  */
-Connector.prototype.id = function (model, prop) {
+Connector.prototype.id = function(model, prop) {
-  var p = this._models[model].properties[prop];
+  var p = this.getModelDefinition(model).properties[prop];
   return p && p.id;
 };
 
@@ -111,10 +163,8 @@ Connector.prototype.id = function (model, prop) {
  * Hook to be called by DataSource for defining a model
  * @param {Object} modelDefinition The model definition
  */
-Connector.prototype.define = function (modelDefinition) {
+Connector.prototype.define = function(modelDefinition) {
-  if (!modelDefinition.settings) {
+  modelDefinition.settings = modelDefinition.settings || {};
-    modelDefinition.settings = {};
-  }
   this._models[modelDefinition.model.modelName] = modelDefinition;
 };
 
@@ -122,18 +172,22 @@ Connector.prototype.define = function (modelDefinition) {
  * Hook to be called by DataSource for defining a model property
  * @param {String} model The model name
  * @param {String} propertyName The property name
- * @param {Object} propertyDefinition The object for property metadata
+ * @param {Object} propertyDefinition The object for property definition
  */
-Connector.prototype.defineProperty = function (model, propertyName, propertyDefinition) {
+Connector.prototype.defineProperty = function(model, propertyName, propertyDefinition) {
-  this._models[model].properties[propertyName] = propertyDefinition;
+    var modelDef = this.getModelDefinition(model);
-};
+    modelDef.properties[propertyName] = propertyDefinition;
+  };
 
 /**
  * Disconnect from the connector
+ * @param {Function} [cb] Callback function
  */
 Connector.prototype.disconnect = function disconnect(cb) {
   // NO-OP
-  if (cb) process.nextTick(cb);
+  if (cb) {
+    process.nextTick(cb);
+  }
 };
 
 /**
@@ -143,7 +197,7 @@ Connector.prototype.disconnect = function disconnect(cb) {
  * @returns {*} The id value
  *
  */
-Connector.prototype.getIdValue = function (model, data) {
+Connector.prototype.getIdValue = function(model, data) {
   return data && data[this.idName(model)];
 };
 
@@ -154,16 +208,69 @@ Connector.prototype.getIdValue = function (model, data) {
  * @param {*} value The id value
  *
  */
-Connector.prototype.setIdValue = function (model, data, value) {
+Connector.prototype.setIdValue = function(model, data, value) {
   if (data) {
     data[this.idName(model)] = value;
   }
 };
 
-Connector.prototype.getType = function () {
+/**
-  return this.type;
+ * Test if a property is nullable
+ * @param {Object} prop The property definition
+ * @returns {boolean} true if nullable
+ */
+Connector.prototype.isNullable = function(prop) {
+  if (prop.required || prop.id) {
+    return false;
+  }
+  if (prop.nullable || prop['null'] || prop.allowNull) {
+    return true;
+  }
+  if (prop.nullable === false || prop['null'] === false ||
+    prop.allowNull === false) {
+    return false;
+  }
+  return true;
 };
 
+/**
+ * Return the DataAccessObject interface implemented by the connector
+ * @returns {Object} An object containing all methods implemented by the
+ * connector that can be mixed into the model class. It should be considered as
+ * the interface.
+ */
+Connector.prototype.getDataAccessObject = function() {
+  return this.DataAccessObject;
+};
+
+/*!
+ * Define aliases to a prototype method/property
+ * @param {Function} cls The class that owns the method/property
+ * @param {String} methodOrPropertyName The official property method/property name
+ * @param {String|String[]} aliases Aliases to the official property/method
+ */
+Connector.defineAliases = function(cls, methodOrPropertyName, aliases) {
+  if (typeof aliases === 'string') {
+    aliases = [aliases];
+  }
+  if (Array.isArray(aliases)) {
+    aliases.forEach(function(alias) {
+      if (typeof alias === 'string') {
+        Object.defineProperty(cls, alias, {
+          get: function() {
+            return this[methodOrPropertyName];
+          }
+        });
+      }
+    });
+  }
+};
+
+/**
+ * `command()` and `query()` are aliases to `execute()`
+ */
+Connector.defineAliases(Connector.prototype, 'execute', ['command', 'query']);
+
 
 
 

lib/parameterized-sql.js

@@ -0,0 +1,100 @@
+var assert = require('assert');
+var PLACEHOLDER = '?';
+
+module.exports = ParameterizedSQL;
+
+/**
+ * A class for parameterized SQL clauses
+ * @param {String|Object} sql The SQL clause. If the value is a string, treat
+ * it as the template using `?` as the placeholder, for example, `(?,?)`. If
+ * the value is an object, treat it as {sql: '...', params: [...]}
+ * @param {*[]} params An array of parameter values. The length should match the
+ * number of placeholders in the template
+ * @returns {ParameterizedSQL} A new instance of ParameterizedSQL
+ * @constructor
+ */
+function ParameterizedSQL(sql, params) {
+  if (!(this instanceof ParameterizedSQL)) {
+    return new ParameterizedSQL(sql, params);
+  }
+  sql = sql || '';
+  if (arguments.length === 1 && typeof sql === 'object') {
+    this.sql = sql.sql;
+    this.params = sql.params || [];
+  } else {
+    this.sql = sql;
+    this.params = params || [];
+  }
+  assert(typeof this.sql === 'string', 'sql must be a string');
+  assert(Array.isArray(this.params), 'params must be an array');
+
+  var parts = this.sql.split(PLACEHOLDER);
+  assert(parts.length - 1 === this.params.length,
+    'The number of ? (' + (parts.length - 1) +
+    ') in the sql (' + this.sql + ') must match the number of params (' +
+    this.params.length +
+    ') ' + this.params);
+}
+
+/**
+ * Merge the parameterized sqls into the current instance
+ * @param {Object|Object[]} ps A parametered SQL or an array of parameterized
+ * SQLs
+ * @param {String} [separator] Separator, default to ` `
+ * @returns {ParameterizedSQL} The current instance
+ */
+ParameterizedSQL.prototype.merge = function(ps, separator) {
+  if (Array.isArray(ps)) {
+    return this.constructor.append(this,
+      this.constructor.join(ps, separator), separator);
+  } else {
+    return this.constructor.append(this, ps, separator);
+  }
+};
+
+ParameterizedSQL.prototype.toJSON = function() {
+  return {
+    sql: this.sql,
+    params: this.params
+  };
+};
+
+/**
+ * Append the statement into the current statement
+ * @param {Object} currentStmt The current SQL statement
+ * @param {Object} stmt The statement to be appended
+ * @param {String} [separator] Separator, default to ` `
+ * @returns {*} The merged statement
+ */
+ParameterizedSQL.append = function(currentStmt, stmt, separator) {
+  currentStmt = (currentStmt instanceof ParameterizedSQL) ?
+    currentStmt : new ParameterizedSQL(currentStmt);
+  stmt = (stmt instanceof ParameterizedSQL) ? stmt :
+    new ParameterizedSQL(stmt);
+  separator = typeof separator === 'string' ? separator : ' ';
+  if (currentStmt.sql) {
+    currentStmt.sql += separator;
+  }
+  if (stmt.sql) {
+    currentStmt.sql += stmt.sql;
+  }
+  currentStmt.params = currentStmt.params.concat(stmt.params);
+  return currentStmt;
+};
+
+/**
+ * Join multiple parameterized SQLs into one
+ * @param {Object[]} sqls An array of parameterized SQLs
+ * @param {String} [separator] Separator, default to ` `
+ * @returns {ParameterizedSQL}
+ */
+ParameterizedSQL.join = function(sqls, separator) {
+  assert(Array.isArray(sqls), 'sqls must be an array');
+  var ps = new ParameterizedSQL('', []);
+  for (var i = 0, n = sqls.length; i < n; i++) {
+    this.append(ps, sqls[i], separator);
+  }
+  return ps;
+};
+
+ParameterizedSQL.PLACEHOLDER = PLACEHOLDER;

package.json

@@ -21,12 +21,13 @@
     "url": "https://github.com/strongloop/loopback-connector/blob/master/LICENSE"
   },
   "dependencies": {
-    "async": "^0.9.0"
+    "async": "^0.9.0",
+    "debug": "^2.1.3"
   },
   "devDependencies": {
-    "chai": "~1.9.2",
+    "chai": "~2.3.0",
-    "jshint": "^2.6.0",
+    "jshint": "^2.7.0",
-    "loopback-datasource-juggler": "^2.0.0",
+    "loopback-datasource-juggler": "^2.26.3",
-    "mocha": "^1.19.0"
+    "mocha": "^2.2.4"
   }
 }

test/connectors/test-sql-connector.js

@@ -2,12 +2,12 @@
  * A mockup connector that extends SQL connector
  */
 var util = require('util');
-var SqlConnector = require('../../lib/sql');
+var SQLConnector = require('../../lib/sql');
 
 exports.initialize = function initializeDataSource(dataSource, callback) {
   process.nextTick(function() {
-    if(callback) {
+    if (callback) {
-      var connector = new TestConnector();
+      var connector = new TestConnector(dataSource.settings);
       connector.dataSource = dataSource;
       dataSource.connector = connector;
       callback(null, connector);
@@ -15,12 +15,65 @@ exports.initialize = function initializeDataSource(dataSource, callback) {
   });
 };
 
-function TestConnector() {
+function TestConnector(settings) {
-  SqlConnector.apply(this, [].slice.call(arguments));
+  SQLConnector.call(this, 'testdb', settings);
   this._tables = {};
 }
 
-util.inherits(TestConnector, SqlConnector);
+util.inherits(TestConnector, SQLConnector);
+
+TestConnector.prototype.escapeName = function(name) {
+  return '`' + name + '`';
+};
+
+TestConnector.prototype.dbName = function(name) {
+  return name.toUpperCase();
+};
+
+TestConnector.prototype.getPlaceholderForValue = function(key) {
+  return '$' + key;
+};
+
+TestConnector.prototype.escapeValue = function(value) {
+  if (typeof value === 'number' || typeof value === 'boolean') {
+    return value;
+  }
+  if (typeof value === 'string') {
+    return "'" + value + "'";
+  }
+  if (value == null) {
+    return 'NULL';
+  }
+  if (typeof value === 'object') {
+    return String(value);
+  }
+  return value;
+};
+
+TestConnector.prototype.toColumnValue = function(prop, val) {
+  return val;
+};
+
+TestConnector.prototype._buildLimit = function(model, limit, offset) {
+  if (isNaN(limit)) {
+    limit = 0;
+  }
+  if (isNaN(offset)) {
+    offset = 0;
+  }
+  if (!limit && !offset) {
+    return '';
+  }
+  return 'LIMIT ' + (offset ? (offset + ',' + limit) : limit);
+};
+
+TestConnector.prototype.applyPagination =
+  function(model, stmt, filter) {
+    /*jshint unused:false */
+    var limitClause = this._buildLimit(model, filter.limit,
+      filter.offset || filter.skip);
+    return stmt.merge(limitClause);
+  };
 
 TestConnector.prototype.dropTable = function(model, cb) {
   var err;
@@ -47,3 +100,7 @@ TestConnector.prototype.createTable = function(model, cb) {
     cb(err);
   });
 };
+
+TestConnector.prototype.executeSQL = function(sql, params, options, callback) {
+  callback(null, []);
+};

test/smoke.test.js

@@ -9,4 +9,22 @@ describe('loopback-connector', function() {
   it('exports SqlConnector', function() {
     assert(connector.SqlConnector);
   });
+
+  it('exports SQLConnector', function() {
+    assert(connector.SQLConnector);
+  });
+
+  it('creates aliases to Connector.prototype.execute', function() {
+    assert.equal(connector.Connector.prototype.execute,
+      connector.Connector.prototype.query);
+    assert.equal(connector.Connector.prototype.execute,
+      connector.Connector.prototype.command);
+  });
+
+  it('creates aliases to SQLConnector.prototype.execute', function() {
+    assert.equal(connector.SQLConnector.prototype.execute,
+      connector.SQLConnector.prototype.query);
+    assert.equal(connector.SQLConnector.prototype.execute,
+      connector.SQLConnector.prototype.command);
+  });
 });

test/sql.test.js

@@ -0,0 +1,310 @@
+var expect = require('chai').expect;
+var SQLConnector = require('../lib/sql');
+var ParameterizedSQL = SQLConnector.ParameterizedSQL;
+var testConnector = require('./connectors/test-sql-connector');
+
+var juggler = require('loopback-datasource-juggler');
+var ds = new juggler.DataSource({
+  connector: testConnector,
+  debug: true
+});
+var connector;
+
+describe('sql connector', function() {
+  before(function() {
+    connector = ds.connector;
+    connector._tables = {};
+    connector._models = {};
+    ds.createModel('customer',
+      {
+        name: {
+          id: true,
+          type: String,
+          testdb: {
+            column: 'NAME',
+            dataType: 'VARCHAR',
+            dataLength: 32
+          }
+        }, vip: {
+        type: Boolean,
+        testdb: {
+          column: 'VIP'
+        }
+      },
+        address: String
+      },
+      {testdb: {table: 'CUSTOMER'}});
+  });
+
+  it('should map table name', function() {
+    var table = connector.table('customer');
+    expect(table).to.eql('CUSTOMER');
+  });
+
+  it('should map column name', function() {
+    var column = connector.column('customer', 'name');
+    expect(column).to.eql('NAME');
+  });
+
+  it('should find column metadata', function() {
+    var column = connector.columnMetadata('customer', 'name');
+    expect(column).to.eql({
+      column: 'NAME',
+      dataType: 'VARCHAR',
+      dataLength: 32
+    });
+  });
+
+  it('should map property name', function() {
+    var prop = connector.propertyName('customer', 'NAME');
+    expect(prop).to.eql('name');
+  });
+
+  it('should map id column name', function() {
+    var idCol = connector.idColumn('customer');
+    expect(idCol).to.eql('NAME');
+  });
+
+  it('should find escaped id column name', function() {
+    var idCol = connector.idColumnEscaped('customer');
+    expect(idCol).to.eql('`NAME`');
+  });
+
+  it('should find escaped table name', function() {
+    var table = connector.tableEscaped('customer');
+    expect(table).to.eql('`CUSTOMER`');
+  });
+
+  it('should find escaped column name', function() {
+    var column = connector.columnEscaped('customer', 'vip');
+    expect(column).to.eql('`VIP`');
+  });
+
+  it('should convert to escaped id column value', function() {
+    var column = connector.idColumnValue('customer', 'John');
+    expect(column).to.eql('John');
+  });
+
+  it('builds where', function() {
+    var where = connector.buildWhere('customer', {name: 'John'});
+    expect(where.toJSON()).to.eql({
+      sql: 'WHERE `NAME`=?',
+      params: ['John']
+    });
+  });
+
+  it('builds where with null', function() {
+    var where = connector.buildWhere('customer', {name: null});
+    expect(where.toJSON()).to.eql({
+      sql: 'WHERE `NAME` IS NULL',
+      params: []
+    });
+  });
+
+  it('builds where with inq', function() {
+    var where = connector.buildWhere('customer', {name: {inq: ['John', 'Mary']}});
+    expect(where.toJSON()).to.eql({
+      sql: 'WHERE `NAME` IN (?,?)',
+      params: ['John', 'Mary']
+    });
+  });
+
+  it('builds where with or', function() {
+    var where = connector.buildWhere('customer',
+      {or: [{name: 'John'}, {name: 'Mary'}]});
+    expect(where.toJSON()).to.eql({
+      sql: 'WHERE (`NAME`=?) OR (`NAME`=?)',
+      params: ['John', 'Mary']
+    });
+  });
+
+  it('builds where with and', function() {
+    var where = connector.buildWhere('customer',
+      {and: [{name: 'John'}, {vip: true}]});
+    expect(where.toJSON()).to.eql({
+      sql: 'WHERE (`NAME`=?) AND (`VIP`=?)',
+      params: ['John', true]
+    });
+  });
+
+  it('builds order by with one field', function() {
+    var orderBy = connector.buildOrderBy('customer', 'name');
+    expect(orderBy).to.eql('ORDER BY `NAME`');
+  });
+
+  it('builds order by with two fields', function() {
+    var orderBy = connector.buildOrderBy('customer', ['name', 'vip']);
+    expect(orderBy).to.eql('ORDER BY `NAME`,`VIP`');
+  });
+
+  it('builds order by with two fields and dirs', function() {
+    var orderBy = connector.buildOrderBy('customer', ['name ASC', 'vip DESC']);
+    expect(orderBy).to.eql('ORDER BY `NAME` ASC,`VIP` DESC');
+  });
+
+  it('builds fields for columns', function() {
+    var fields = connector.buildFields('customer', {name: 'John', vip: true});
+    expect(fields.names).to.eql(['`NAME`', '`VIP`']);
+    expect(fields.columnValues[0].toJSON()).to.eql(
+      {sql: '?', params: ['John']});
+    expect(fields.columnValues[1].toJSON()).to.eql(
+      {sql: '?', params: [true]});
+  });
+
+  it('builds fields for UPDATE without ids', function() {
+    var fields = connector.buildFieldsForUpdate('customer',
+      {name: 'John', vip: true});
+    expect(fields.toJSON()).to.eql({
+      sql: 'SET `VIP`=?',
+      params: [true]
+    });
+  });
+
+  it('builds fields for UPDATE with ids', function() {
+    var fields = connector.buildFieldsForUpdate('customer',
+      {name: 'John', vip: true}, false);
+    expect(fields.toJSON()).to.eql({
+      sql: 'SET `NAME`=?,`VIP`=?',
+      params: ['John', true]
+    });
+  });
+
+  it('builds column names for SELECT', function() {
+    var cols = connector.buildColumnNames('customer');
+    expect(cols).to.eql('`NAME`,`VIP`,`ADDRESS`');
+  });
+
+  it('builds column names with true fields filter for SELECT', function() {
+    var cols = connector.buildColumnNames('customer', {fields: {name: true}});
+    expect(cols).to.eql('`NAME`');
+  });
+
+  it('builds column names with false fields filter for SELECT', function() {
+    var cols = connector.buildColumnNames('customer', {fields: {name: false}});
+    expect(cols).to.eql('`VIP`,`ADDRESS`');
+  });
+
+  it('builds column names with array fields filter for SELECT', function() {
+    var cols = connector.buildColumnNames('customer', {fields: ['name']});
+    expect(cols).to.eql('`NAME`');
+  });
+
+  it('builds DELETE', function() {
+    var sql = connector.buildDelete('customer', {name: 'John'});
+    expect(sql.toJSON()).to.eql({
+      sql: 'DELETE FROM `CUSTOMER` WHERE `NAME`=$1',
+      params: ['John']
+    });
+  });
+
+  it('builds UPDATE', function() {
+    var sql = connector.buildUpdate('customer', {name: 'John'}, {vip: false});
+    expect(sql.toJSON()).to.eql({
+      sql: 'UPDATE `CUSTOMER` SET `VIP`=$1 WHERE `NAME`=$2',
+      params: [false, 'John']
+    });
+  });
+
+  it('builds SELECT', function() {
+    var sql = connector.buildSelect('customer',
+      {order: 'name', limit: 5, where: {name: 'John'}});
+    expect(sql.toJSON()).to.eql({
+      sql: 'SELECT `NAME`,`VIP`,`ADDRESS` FROM `CUSTOMER`' +
+      ' WHERE `NAME`=$1 ORDER BY `NAME` LIMIT 5',
+      params: ['John']
+    });
+  });
+
+  it('builds INSERT', function() {
+    var sql = connector.buildInsert('customer', {name: 'John', vip: true});
+    expect(sql.toJSON()).to.eql({
+      sql: 'INSERT INTO `CUSTOMER`(`NAME`,`VIP`) VALUES($1,$2)',
+      params: ['John', true]
+    });
+  });
+
+  it('normalizes a SQL statement from string', function() {
+    var sql = 'SELECT * FROM `CUSTOMER`';
+    var stmt = new ParameterizedSQL(sql);
+    expect(stmt.toJSON()).to.eql({sql: sql, params: []});
+  });
+
+  it('normalizes a SQL statement from object without params', function() {
+    var sql = {sql: 'SELECT * FROM `CUSTOMER`'};
+    var stmt = new ParameterizedSQL(sql);
+    expect(stmt.toJSON()).to.eql({sql: sql.sql, params: []});
+  });
+
+  it('normalizes a SQL statement from object with params', function() {
+    var sql =
+    {sql: 'SELECT * FROM `CUSTOMER` WHERE `NAME`=?', params: ['John']};
+    var stmt = new ParameterizedSQL(sql);
+    expect(stmt.toJSON()).to.eql({sql: sql.sql, params: ['John']});
+  });
+
+  it('should throw if the statement is not a string or object', function() {
+    expect(function() {
+      /*jshint unused:false */
+      var stmt = new ParameterizedSQL(true);
+    }).to.throw('sql must be a string');
+  });
+
+  it('concats SQL statements', function() {
+    var stmt1 = {sql: 'SELECT * from `CUSTOMER`'};
+    var where = {sql: 'WHERE `NAME`=?', params: ['John']};
+    stmt1 = ParameterizedSQL.append(stmt1, where);
+    expect(stmt1.toJSON()).to.eql(
+      {sql: 'SELECT * from `CUSTOMER` WHERE `NAME`=?', params: ['John']});
+  });
+
+  it('concats string SQL statements', function() {
+    var stmt1 = 'SELECT * from `CUSTOMER`';
+    var where = {sql: 'WHERE `NAME`=?', params: ['John']};
+    stmt1 = ParameterizedSQL.append(stmt1, where);
+    expect(stmt1.toJSON()).to.eql(
+      {sql: 'SELECT * from `CUSTOMER` WHERE `NAME`=?', params: ['John']});
+  });
+
+  it('should throw if params does not match placeholders', function() {
+    expect(function() {
+      var stmt1 = 'SELECT * from `CUSTOMER`';
+      var where = {sql: 'WHERE `NAME`=?', params: ['John', 'Mary']};
+      stmt1 = ParameterizedSQL.append(stmt1, where);
+    }).to.throw('must match the number of params');
+  });
+
+  it('should allow execute(sql, callback)', function(done) {
+    connector.execute('SELECT * FROM `CUSTOMER`', done);
+  });
+
+  it('should allow execute(sql, params, callback)', function(done) {
+    connector.execute('SELECT * FROM `CUSTOMER` WHERE `NAME`=$1',
+      ['xyz'], done);
+  });
+
+  it('should allow execute(sql, params, options, callback)', function(done) {
+    connector.execute('SELECT * FROM `CUSTOMER` WHERE `NAME`=$1',
+      ['xyz'], {transaction: true}, done);
+  });
+
+  it('should throw if params is not an array for execute()', function() {
+    expect(function() {
+      connector.execute('SELECT * FROM `CUSTOMER`', 'xyz', function() {
+      });
+    }).to.throw('params must be an array');
+  });
+
+  it('should throw if options is not an object for execute()', function() {
+    expect(function() {
+      connector.execute('SELECT * FROM `CUSTOMER`', [], 'xyz', function() {
+      });
+    }).to.throw('options must be an object');
+  });
+
+  it('should throw if callback is not a function for execute()', function() {
+    expect(function() {
+      connector.execute('SELECT * FROM `CUSTOMER`', [], {}, 'xyz');
+    }).to.throw('callback must be a function');
+  });
+});
+