loopback-connector-mysql/README.md

986 lines
23 KiB
Markdown
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# loopback-connector-mysql
[MySQL](https://www.mysql.com/) is a popular open-source relational database
management system (RDBMS). The `loopback-connector-mysql` module provides the
MySQL connector module for the LoopBack framework.
## Installation
In your application root directory, enter this command to install the connector:
```sh
npm install loopback-connector-mysql --save
```
**Note**: Since `loopback-connector-mysql` v7.x.x, this MySQL connector has dropped support for MySQL 5.7 and requires MySQL 8.0+.
This installs the module from npm and adds it as a dependency to the
application's `package.json` file.
If you create a MySQL data source using the data source generator as described
below, you don't have to do this, since the generator will run `npm install` for
you.
## Creating a MySQL data source
For LoopBack 4 users, use the LoopBack 4
[Command-line interface](https://loopback.io/doc/en/lb4/Command-line-interface.html)
to generate a DataSource with MySQL connector to your LB4 application. Run
[`lb4 datasource`](https://loopback.io/doc/en/lb4/DataSource-generator.html), it
will prompt for configurations such as host, post, etc. that are required to
connect to a MySQL database.
After setting it up, the configuration can be found under
`src/datasources/<DataSourceName>.datasource.ts`, which would look like this:
```ts
const config = {
name: 'db',
connector: 'mysql',
url: '',
host: 'localhost',
port: 3306,
user: 'user',
password: 'pass',
database: 'testdb',
};
```
<details><summary markdown="span"><strong>For LoopBack 3 users</strong></summary>
Use
the [Data source generator](http://loopback.io/doc/en/lb3/Data-source-generator.html) to
add a MySQL data source to your application.
The generator will prompt for the database server hostname, port, and other
settings required to connect to a MySQL database. It will also run the
`npm install` command above for you.
The entry in the application's `/server/datasources.json` will look like this:
```javascript
"mydb": {
"name": "mydb",
"connector": "mysql",
"host": "myserver",
"port": 3306,
"database": "mydb",
"password": "mypassword",
"user": "admin"
}
```
</details>
Edit `<DataSourceName>.datasources.ts` to add any other additional properties
that you require.
### Properties
<table>
<thead>
<tr>
<th width="150">Property</th>
<th width="80">Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>collation</td>
<td>String</td>
<td>Determines the charset for the connection. Default is utf8_general_ci.</td>
</tr>
<tr>
<td>connector</td>
<td>String</td>
<td>Connector name, either “loopback-connector-mysql” or “mysql”.</td>
</tr>
<tr>
<td>connectionLimit</td>
<td>Number</td>
<td>The maximum number of connections to create at once. Default is 10.</td>
</tr>
<tr>
<td>database</td>
<td>String</td>
<td>Database name</td>
</tr>
<tr>
<td>debug</td>
<td>Boolean</td>
<td>If true, turn on verbose mode to debug database queries and lifecycle.</td>
</tr>
<tr>
<td>host</td>
<td>String</td>
<td>Database host name</td>
</tr>
<tr>
<td>password</td>
<td>String</td>
<td>Password to connect to database</td>
</tr>
<tr>
<td>port</td>
<td>Number</td>
<td>Database TCP port</td>
</tr>
<tr>
<td>socketPath</td>
<td>String</td>
<td>The path to a unix domain socket to connect to. When used host and port are ignored.</td>
</tr>
<tr>
<td>supportBigNumbers</td>
<td>Boolean</td>
<td>Enable this option to deal with big numbers (BIGINT and DECIMAL columns) in the database. Default is false.</td>
</tr>
<tr>
<td>timeZone</td>
<td>String</td>
<td>The timezone used to store local dates. Default is local.</td>
</tr>
<tr>
<td>url</td>
<td>String</td>
<td>Connection URL of form <code>mysql://user:password@host/db</code>. Overrides other connection settings.</td>
</tr>
<tr>
<td>username</td>
<td>String</td>
<td>Username to connect to database</td>
</tr>
<tr>
<td>allowExtendedOperators</td>
<td>Boolean</td>
<td>Set to <code>true</code> to enable MySQL-specific operators
such as <code>match</code>. Learn more in
<a href="#extended-operators">Extended operators</a> below.
</td>
</tr>
</tbody>
</table>
**NOTE**: In addition to these properties, you can use additional parameters
supported by [`node-mysql`](https://github.com/felixge/node-mysql).
## Type mappings
See [LoopBack 4 types](http://loopback.io/doc/en/lb4/LoopBack-types.html) (or [LoopBack 3 types](http://loopback.io/doc/en/lb3/LoopBack-types.html)) for
details on LoopBack's data types.
### LoopBack to MySQL types
<table>
<thead>
<tr>
<th width="450">LoopBack Type</th>
<th width="450">MySQL Type</th>
</tr>
</thead>
<tbody>
<tr>
<td>String/JSON</td>
<td>VARCHAR</td>
</tr>
<tr>
<td>Text</td>
<td>TEXT</td>
</tr>
<tr>
<td>Number</td>
<td>INT</td>
</tr>
<tr>
<td>Date</td>
<td>DATETIME</td>
</tr>
<tr>
<td>Boolean</td>
<td>TINYINT(1)</td>
</tr>
<tr>
<td><a href="http://apidocs.strongloop.com/loopback-datasource-juggler/#geopoint" class="external-link">GeoPoint</a> object</td>
<td>POINT</td>
</tr>
<tr>
<td>Custom Enum type<br>(See <a href="#enum">Enum</a> below)</td>
<td>ENUM</td>
</tr>
</tbody>
</table>
### MySQL to LoopBack types
<table>
<thead>
<tr>
<th width="450">MySQL Type</th>
<th width="450">LoopBack Type</th>
</tr>
</thead>
<tbody>
<tr>
<td>CHAR</td>
<td>String</td>
</tr>
<tr>
<td>BIT(1)<br>CHAR(1)<br>TINYINT(1)</td>
<td>Boolean</td>
</tr>
<tr>
<td>VARCHAR<br>TINYTEXT<br>MEDIUMTEXT<br>LONGTEXT<br>TEXT<br>ENUM<br>SET</td>
<td>String</td>
</tr>
<tr>
<td>TINYBLOB<br>MEDIUMBLOB<br>LONGBLOB<br>BLOB<br>BINARY<br>VARBINARY<br>BIT</td>
<td>Node.js <a href="http://nodejs.org/api/buffer.html">Buffer object</a></td>
</tr>
<tr>
<td>TINYINT<br>SMALLINT<br>INT<br>MEDIUMINT<br>YEAR<br>FLOAT<br>DOUBLE<br>NUMERIC<br>DECIMAL</td>
<td>
<p>Number<br>For FLOAT and DOUBLE, see <a href="#floating-point-types">Floating-point types</a>. </p>
<p>For NUMERIC and DECIMAL, see <a href="MySQL-connector.html">Fixed-point exact value types</a></p>
</td>
</tr>
<tr>
<td>DATE<br>TIMESTAMP<br>DATETIME</td>
<td>Date</td>
</tr>
</tbody>
</table>
_NOTE_ as of v3.0.0 of MySQL Connector, the following flags were introduced:
- `treatCHAR1AsString` default `false` - treats CHAR(1) as a String instead of a
Boolean
- `treatBIT1AsBit` default `true` - treats BIT(1) as a Boolean instead of a
Binary
- `treatTINYINT1AsTinyInt` default `true` - treats TINYINT(1) as a Boolean
instead of a Number
## Data mapping properties
Except the common database-specific properties we introduce in [How LoopBack Models Map To Database Tables/Collections](https://loopback.io/doc/en/lb4/Model.html#how-loopback-models-map-to-database-tablescollections), the following are more detailed examples and MySQL-specific settings.
### Table/Column Names
Besides the basic LoopBack types, as we introduced above, you can also specify
additional MySQL-specific properties for a LoopBack model. It would be mapped to
the database.
Use the `mysql.<property>` in the model definition or the property definition to
configure the table/column definition.
For example, the following settings would allow you to have custom table name
(`Custom_User`) and column name (`custom_id` and `custom_name`). Such mapping is
useful when you'd like to have different table/column names from the model:
{% include code-caption.html content="user.model.ts" %}
```ts
@model({
settings: { mysql: { schema: 'testdb', table: 'Custom_User'} },
})
export class User extends Entity {
@property({
type: 'number',
required: true,
id: true,
mysql: {
columnName: 'custom_id',
},
})
id: number;
@property({
type: 'string',
mysql: {
columnName: 'custom_name',
},
})
name?: string;
```
<details><summary markdown="span"><strong>For LoopBack 3 users</strong></summary>
```javascript
{
"name": "User",
"options": {
"mysql": {
"schema": "testdb",
"table": "Custom_User"
}
},
"properties": {
"id": {
"type": "Number",
"required": true,
"mysql": {
"columnName": "custom_id",
}
},
"name": {
"type": "String",
"mysql": {
"columnName": "custom_name",
}
},
}
}
```
</details>
### Numeric Types
Except the names, you can also use the dataType column/property attribute to
specify what MySQL column type to use. The following MySQL type-dataType
combinations are supported:
- number
- integer
- tinyint
- smallint
- mediumint
- int
- bigint
- float
- double
- decimal
The following examples will be in LoopBack 4 style, but it's the same if you
provide `mysql.<property>` to the LB3 property definition.
#### Floating-point types
For Float and Double data types, use the `precision` and `scale` options to
specify custom precision. Default is (16,8).
<details><summary markdown="span"><strong>Example</strong></summary>
```ts
@property({
type: 'Number',
mysql: {
dataType: 'float',
precision: 20,
scale: 4
}
})
price: Number;
```
</details>
#### Fixed-point exact value types
For Decimal and Numeric types, use the `precision` and `scale` options to
specify custom precision. Default is (9,2). These aren't likely to function as
true fixed-point.
<details><summary markdown="span"><strong>Example</strong></summary>
```ts
@property({
type: 'Number',
mysql: {
dataType: 'decimal',
precision: 12,
scale: 8
}
})
price: Number;
```
</details>
### Text types
Convert String / DataSource.Text / DataSource.JSON to the following MySQL types:
- varchar
- char
- text
- mediumtext
- tinytext
- longtext
<details><summary markdown="span"><strong>Example</strong></summary>
```ts
@property({
type: 'String',
mysql: {
dataType: 'char',
dataLength: 24 // limits the property length
},
})
userName: String;
```
</details>
### Dat types
Convert JSON Date types to datetime or timestamp.
<details><summary markdown="span"><strong>Example</strong></summary>
```ts
@property({
type: 'Date',
mysql: {
dataType: 'timestamp',
},
})
startTime: Date;
```
</details>
### Enum
See the [Model ENUM property](https://loopback.io/doc/en/lb4/Model.html#enum-property) for details.
### Default Clause/Constant
Use the `default` and `dataType` properties to have MySQL handle **setting column `DEFAULT` value**.
<details><summary markdown="span"><strong>Example</strong></summary>
```ts
@property({
type: 'String',
mysql: {
dataType: 'varchar',
default: 'pending'
}
})
status: String;
@property({
type: 'Number',
mysql: {
dataType: 'int',
default: 42
}
})
maxDays: Number;
@property({
type: 'boolean',
mysql: {
dataType: 'tinyint',
default: 1
}
})
isDone: Boolean;
```
</details>
For the date or timestamp types use `CURRENT_TIMESTAMP` or `now`.
<details><summary markdown="span"><strong>Example</strong></summary>
```ts
@property({
type: 'Date',
mysql: {
dataType: 'datetime',
default: 'CURRENT_TIMESTAMP'
}
})
last_modified: Date;
```
</details>
**NOTE**: The following column types do **NOT** supported
[MySQL Default Values](https://dev.mysql.com/doc/refman/5.7/en/data-type-defaults.html):
- BLOB
- TEXT
- GEOMETRY
- JSON
## Extended operators
MySQL connector supports the following MySQL-specific operators:
- [`match`](#operator-match)
Please note extended operators are disabled by default, you must enable
them at datasource level or model level by setting `allowExtendedOperators` to
`true`.
### Operator `match`
The `match` operator allows you to perform a full text search using the [MATCH() .. AGAINST()](https://dev.mysql.com/doc/refman/8.0/en/fulltext-search.html) operator in MySQL.
Three different modes of the `MATCH` clause are also available in the form of operators -
- `matchbool` for [Boolean Full Text Search](https://dev.mysql.com/doc/refman/8.0/en/fulltext-boolean.html)
- `matchnl` for [Natural Language Full Text Search](https://dev.mysql.com/doc/refman/8.0/en/fulltext-natural-language.html)
- `matchqe` for [Full-Text Searches with Query Expansion](https://dev.mysql.com/doc/refman/8.0/en/fulltext-query-expansion.html)
- `matchnlqe` for [Full-Text Searches with Query Expansion](https://dev.mysql.com/doc/refman/8.0/en/fulltext-query-expansion.html) with the `IN NATURAL LANGUAGE MODE WITH QUERY EXPANSION` modifier.
By default, the `match` operator works in Natural Language mode.
**Note** The fields you are querying must be setup with a `FULLTEXT` index to perform full text search on them.
Assuming a model such as this:
```ts
@model({
settings: {
allowExtendedOperators: true,
}
})
class Post {
@property({
type: 'string',
mysql: {
index: {
kind: 'FULLTEXT'
}
},
})
content: string;
}
```
You can query the content field as follows:
```ts
const posts = await postRepository.find({
where: {
{
content: {match: 'someString'},
}
}
});
```
## Discovery and auto-migration
### Model discovery
The MySQL connector supports _model discovery_ that enables you to create
LoopBack models based on an existing database schema. Once you defined your
datasource:
- LoopBack 4 users could use the commend
[`lb4 discover`](https://loopback.io/doc/en/lb4/Discovering-models.html) to
discover models.
- For LB3 users, please check
[Discovering models from relational databases](https://loopback.io/doc/en/lb3/Discovering-models-from-relational-databases.html).
(See
[database discovery API](http://apidocs.strongloop.com/loopback-datasource-juggler/#datasource-prototype-discoverandbuildmodels)
for related APIs information)
### Auto-migration
The MySQL connector also supports _auto-migration_ that enables you to create a
database schema from LoopBack models. For example, based on the following model,
the auto-migration method would create/alter existing `Customer` table in the
database. Table `Customer` would have two columns: `name` and `id`, where `id`
is also the primary key that has `auto_increment` set as it has definition of
`type: 'Number'` and `generated: true`:
```ts
@model()
export class Customer extends Entity {
@property({
id: true,
type: 'Number',
generated: true,
})
id: number;
@property({
type: 'string',
})
name: string;
}
```
Moreover, additional MySQL-specific properties mentioned in the
[Data mapping properties](#data-mapping-properties) section work with
auto-migration as well.
#### Auto-generated ids
For now LoopBack MySQL connector only supports auto-generated id
(`generated: true`) for integer type as for MySQL, the default id type is
_integer_. If you'd like to use other types such as string (uuid) as the id
type, you can:
- use uuid that is **generated by your LB application** by setting
[`defaultFn: uuid`](https://loopback.io/doc/en/lb4/Model.html#property-decorator).
```ts
@property({
id: true,
type: 'string'
defaultFn: 'uuidv4',
// generated: true, -> not needed
})
id: string;
```
- Alter the table in your database to use a certain function if you prefer
having **the database to generate the value**.
```ts
@property({
id: true,
type: 'string'
generated: true, // to indicate the value generates by the db
useDefaultIdType: false, // needed
})
id: string;
```
#### Auto-migrate/Auto-update models with foreign keys
Foreign key constraints can be defined in the model definition.
**Note**: The order of table creation is important. A referenced table must
exist before creating a foreign key constraint. The order can be specified
using the optional <a href="https://loopback.io/doc/en/lb4/apidocs.repository.schemamigrationoptions.html">`SchemaMigrationOptions`</a> argument of `migrateSchema`:
```
await app.migrateSchema({
models: [ 'Customer', 'Order' ]
});
```
Define your models and the foreign key constraints as follows:
{% include code-caption.html content="customer.model.ts" %}
```ts
@model()
export class Customer extends Entity {
@property({
id: true,
type: 'Number',
generated: true,
})
id: number;
@property({
type: 'string',
})
name: string;
}
```
`order.model.ts`:
```ts
@model({
settings: {
foreignKeys: {
fk_order_customerId: {
name: 'fk_order_customerId',
entity: 'Customer',
entityKey: 'id',
foreignKey: 'customerId',
},
},
})
export class Order extends Entity {
@property({
id: true,
type: 'Number',
generated: true
})
id: number;
@property({
type: 'string'
})
name: string;
@property({
type: 'Number'
})
customerId: number;
}
```
<details><summary markdown="span"><strong>For LoopBack 3 users</strong></summary>
```json
({
"name": "Customer",
"options": {
"idInjection": false
},
"properties": {
"id": {
"type": "Number",
"id": 1
},
"name": {
"type": "String",
"required": false
}
}
},
{
"name": "Order",
"options": {
"idInjection": false,
"foreignKeys": {
"fk_order_customerId": {
"name": "fk_order_customerId",
"entity": "Customer",
"entityKey": "id",
"foreignKey": "customerId"
}
}
},
"properties": {
"id": {
"type": "Number"
"id": 1
},
"customerId": {
"type": "Number"
},
"description": {
"type": "String",
"required": false
}
}
})
```
</details>
MySQL handles the foreign key integrity by the referential action specified by
`ON UPDATE` and `ON DELETE`. You can specify which referential actions the
foreign key follows in the model definition upon auto-migrate or auto-update
operation. Both `onDelete` and `onUpdate` default to `restrict`.
Take the example we showed above, let's add the referential action to the
foreign key `customerId`:
```ts
@model({
settings: {
foreignKeys: {
fk_order_customerId: {
name: 'fk_order_customerId',
entity: 'Customer',
entityKey: 'id',
foreignKey: 'customerId',
onUpdate: 'restrict', // restrict|cascade|set null|no action|set default
onDelete: 'cascade' // restrict|cascade|set null|no action|set default
},
},
})
export class Order extends Entity {
...
```
<details><summary markdown="span"><strong>For LoopBack 3 users</strong></summary>
**model-definiton.json**
```json
{
"name": "Customer",
"options": {
"idInjection": false
},
"properties": {
"id": {
"type": "Number",
"id": 1
},
"name": {
"type": "String",
"required": false
}
}
},
{
"name": "Order",
"options": {
"idInjection": false,
"foreignKeys": {
"fk_order_customerId": {
"name": "fk_order_customerId",
"entity": "Customer",
"entityKey": "id",
"foreignKey": "customerId",
"onUpdate": "restrict",
"onDelete": "cascade"
}
}
},
"properties": {
"id": {
"type": "Number"
"id": 1
},
"customerId": {
"type": "Number"
},
"description": {
"type": "String",
"required": false
}
}
}
```
**boot-script.js**
```js
module.exports = function (app) {
var mysqlDs = app.dataSources.mysqlDS;
var Book = app.models.Order;
var Author = app.models.Customer;
// first autoupdate the `Customer` model to avoid foreign key constraint failure
mysqlDs.autoupdate('Customer', function (err) {
if (err) throw err;
console.log('\nAutoupdated table `Customer`.');
mysqlDs.autoupdate('Order', function (err) {
if (err) throw err;
console.log('\nAutoupdated table `Order`.');
// at this point the database table `Order` should have one foreign key `customerId` integrated
});
});
};
```
</details>
#### Breaking Changes with GeoPoint since 5.x
Prior to `loopback-connector-mysql@5.x`, MySQL connector was saving and loading
GeoPoint properties from the MySQL database in reverse. MySQL expects values to
be `POINT(X, Y)` or `POINT(lng, lat)`, but the connector was saving them in the
opposite order(i.e. `POINT(lat,lng)`).
Use the `geopoint` type to achieve so:
```ts
@property({
type: 'geopoint'
})
name: GeoPoint;
```
If you have an application with a model that has a GeoPoint property using
previous versions of this connector, you can migrate your models using the
following programmatic approach:
<details><summary markdown="span"><strong>Click here to expand</strong></summary>
**NOTE** Please back up the database tables that have your application data
before performing any of the steps.
1. Create a boot script under `server/boot/` directory with the following:
```js
'use strict';
module.exports = function (app) {
function findAndUpdate() {
var teashop = app.models.teashop;
//find all instances of the model we'd like to migrate
teashop.find({}, function (err, teashops) {
teashops.forEach(function (teashopInstance) {
//what we fetch back from the db is wrong, so need to revert it here
var newLocation = {
lng: teashopInstance.location.lat,
lat: teashopInstance.location.lng,
};
//only update the GeoPoint property for the model
teashopInstance.updateAttribute('location', newLocation, function (
err,
inst,
) {
if (err) console.log('update attribute failed', err);
else console.log('updateAttribute successful');
});
});
});
}
findAndUpdate();
};
```
2. Run the boot script by simply running your application or `node .`
For the above example, the model definition is as follows:
```json
{
"name": "teashop",
"base": "PersistedModel",
"idInjection": true,
"options": {
"validateUpsert": true
},
"properties": {
"name": {
"type": "string",
"default": "storename"
},
"location": {
"type": "geopoint"
}
},
"validations": [],
"relations": {},
"acls": [],
"methods": {}
}
```
</details>
## Running tests
### Own instance
If you have a local or remote MySQL instance and would like to use that to run
the test suite, use the following command:
- Linux
```bash
MYSQL_HOST=<HOST> MYSQL_PORT=<PORT> MYSQL_USER=<USER> MYSQL_PASSWORD=<PASSWORD> MYSQL_DATABASE=<DATABASE> CI=true npm test
```
- Windows
```bash
SET MYSQL_HOST=<HOST> SET MYSQL_PORT=<PORT> SET MYSQL_USER=<USER> SET MYSQL_PASSWORD=<PASSWORD> SET MYSQL_DATABASE=<DATABASE> SET CI=true npm test
```
### Docker
If you do not have a local MySQL instance, you can also run the test suite with
very minimal requirements.
- Assuming you have [Docker](https://docs.docker.com/engine/installation/)
installed, run the following script which would spawn a MySQL instance on your
local:
```bash
source setup.sh <HOST> <PORT> <USER> <PASSWORD> <DATABASE>
```
where `<HOST>`, `<PORT>`, `<USER>`, `<PASSWORD>` and `<DATABASE>` are optional
parameters. The default values are `localhost`, `3306`, `root`, `pass` and
`testdb` respectively.
- Run the test:
```bash
npm test
```