Commit Graph

35 Commits

Author SHA1 Message Date
kobaska 7078c5d0e5
Optimise replication
Add a new model-level setting "replicationChunkSize" which allows
users to configure change replication algorithm to issue several
smaller requests to fetch changes and upload updates.
2017-02-22 15:12:54 +01:00
kobaska e15a656714 Add option disabling periodic change rectification
When `Model.settings.changeCleanupInterval` is set to a negative value,
no periodic cleanup is performed at all.
2017-01-04 15:58:44 +01:00
Miroslav Bajtoš 1ec7cf0a37 Turn on "no-unused-expressions" rule for eslint
Fix unit-tests relying on property-based assertions to use function
calls instead, using "dirty-chai" to modify chai's property checkers.
2016-11-22 15:30:04 +01:00
Loay 06cb481c3f Update eslint to loopback config v5
Notable side-effects:
 - loopback no longer exports "caller" and "arguments" properties
 - kv-memory connector is now properly added to the connector registry
 - the file "test/support.js" was finally removed
2016-11-22 14:08:02 +01:00
Kogulan Baskaran b4f1b2f02c Add options to bulkUpdate 2016-11-15 17:40:44 +01:00
Sonali Samantaray aef6dca30c Expose upsertWithWhere method 2016-09-02 18:10:47 +05:30
Miroslav Bajtoš a751230cd9 test: increase timeout to prevent CI failures 2016-08-08 15:45:53 +02:00
Supasate Choochaisri 04e26fae5c Separate error-checking and next/done logic from other logic in the test suite
Signed-off-by: Supasate Choochaisri <supasate.c@gmail.com>
2016-05-05 11:12:48 +07:00
Ryan Graham 6964914bab
update copyright statements 2016-05-03 15:50:21 -07:00
Miroslav Bajtoš f9702b0ace Use eslint with loopback config
Drop jshint and jscs in favour of eslint.

Fix style violations.

While we are at this, reduce the max line length from 150 to 100.
2016-04-06 10:45:30 +02:00
Miroslav Bajtoš 782e89758e test: remove forgotten console.trace logs 2016-02-05 12:39:47 +01:00
Miroslav Bajtoš 91c1df96f0 Fix race condition in replication tests 2016-02-05 12:24:27 +01:00
Amir Jafarian 4aac2776a8 Fix bulkUpdate to not trigger rectifyAll
Fix `getIdFromWhereByModelId()` to correctly detect the situation
when "bulkUpdate" performs a write operation using a where filter
containing both id attribute but also all other model attributes.

This should significantly improve the performance of change replication,
because the cost of running rectifyAll is very high.
2015-12-22 10:44:28 +01:00
Pradnya Baviskar 64a1dbadc8 Promisify 'PersistedModel - replication' 2015-08-12 09:22:53 +02:00
Raymond Feng 20c9695591 Make the test compatible with latest juggler 2015-05-13 12:52:31 -07:00
Miroslav Bajtoš 9c5fe088e3 AccessControl for change replication
1) Add integration tests running change replication over REST to verify
that access control at model level is correctly enforced.

2) Implement a new access type "REPLICATE" that allows principals
to create new checkpoints, even though they don't have full WRITE
access to the model. Together with the "READ" permission, these
two types allow principals to replicate (pull) changes from the server.

Note that anybody having "WRITE" access type is automatically
granted "REPLICATE" type too.

3) Add a new model option "enableRemoteReplication" that exposes
replication methods via strong remoting, but does not configure
change rectification. This option should be used the clients
when setting up Remote models attached to the server via the remoting
connector.
2015-04-07 19:53:58 +02:00
Miroslav Bajtoš 65c14c1779 Add conflict resolution API
New methods:
  conflict.resolveUsingSource(cb)
  conflict.resolveUsingTarget(cb)
  conflict.resolveManually(data, cb)
2015-03-20 17:47:07 +01:00
Miroslav Bajtoš 87940a4b58 Detect 3rd-party changes made during replication
Modify `Change.diff()` to include current data revision in each
delta reported back. The current data revision is stored in
`delta.prev`.

Modify `PersistedModel.bulkUpdate()` to check that the current data
revision matches `delta.prev` and report a conflict if a third party
has modified the database under our hands.

Fix `Change` implementation and tests so that they are no longer
attempting to create instances with duplicate ids.
(This used to work because the memory connector was silently
converting such requests to updateOrCreate/findOrCreate.)
2015-03-20 08:19:59 +01:00
Miroslav Bajtoš 91f59e1ccd Remove unnecessary delay in tests. 2015-03-19 08:00:37 +01:00
Miroslav Bajtoš 59ae90bb97 Run replication tests in the browser too 2015-03-06 14:50:16 +01:00
Miroslav Bajtoš 8f41ac4b35 Add replication tests for conflict resolution 2015-03-06 14:50:16 +01:00
Miroslav Bajtoš f57086d5f2 Fix an assertion broke by recent chai upgrade. 2015-03-06 11:22:30 +01:00
Miroslav Bajtoš e59493ec40 Merge pull request #1176 from strongloop/feature/more-replication-improvements
Prevent more kinds of false replication conflicts
2015-03-06 07:27:16 +01:00
Miroslav Bajtoš c2e1b12644 Add more integration tests for replication
Add tests covering typical replication scenarios that happen
in the setup where multiple clients are synchronizing changes
against a single server (database).
2015-03-05 14:55:55 +01:00
Miroslav Bajtoš 76d9244448 Prevent more kinds of false replication conflicts
Rework the Change model to merge changes made within the same
Checkpoint.

Rework `replicate()` to run multiple iteration until there were no
changes replicated. This ensures that the target model is left in
a clean state with no pending changes associated with the latest
(current) checkpoint.
2015-03-05 14:15:02 +01:00
Raymond Feng 2ce1d14a0e Upgrade deps 2015-03-04 16:30:03 -08:00
Miroslav Bajtoš 628e3a30ca Return new checkpoints in callback of replicate()
Extend `PersistedModel.replicate` to pass the newly created checkpoints
as the third callback argument.

The typical usage of these values is to pass them as the `since`
argument of the next `replicate()` call.

    global.since = -1;

    function sync(cb) {
      LocalModel.replicate(
        since,
        RemoteModel,
        function(err, conflicts, cps)
          if (err) return cb(err);
          if (!conflicts.length) {
            since = cps;
            return cb();
          }
          // resolve conflicts and try again
        });
    }
2015-03-03 19:37:11 +01:00
Miroslav Bajtoš 2dc230b7cf Replication: fix checkpoint-related race condition
Rework the "replicate()" to create a new source checkpoint as the first
step, so that any changes made in parallel will be associated with
the next checkpoint.

Before this commit, there was a race condition where a change might
end up being associated with the already-replicated checkpoint and thus
not picked up by the next replication run.
2015-03-03 19:37:11 +01:00
Miroslav Bajtoš 2f9cf115c3 Support different "since" for source and target
Modify `PersistedModel.replicate` to allow consumers to provide
different "since" values to be used for the local and remote changes.

Make the "since" filters consistent and include the "since" value
in the range via `gte`. Before this commit, the local query used
`gt` and the remote query used `gte`.
2015-03-03 14:40:59 +01:00
Miroslav Bajtoš f21b29e34a test: setup GUID for all models tracking changes
Fix all test models that are tracking changes so that they have
a generated unique string id.

Make model names unique where possible to prevent tests reusing
the same model which causes unintented side effects.
2015-03-02 18:16:18 +01:00
Miroslav Bajtoš 702ecc6f72 Fix change detection & tracking
Add unit-tests to verify that all DAO methods correctly create change
records.

Rework the change detection to use the new operation hooks, this fixes
the bugs where operations like "updateOrCreate" did not update change
records.
2015-02-20 19:28:33 +01:00
Rob Halff 36e1f6840c fix jscs errors 2014-11-21 03:35:36 +01:00
Miroslav Bajtoš ea5b9d16fc Rename DataModel to PersistedModel 2014-06-05 09:56:00 +02:00
Ritchie Martori eec7bdd5f4 - Use the RemoteObjects class to find remote objects instead of
creating a cache
 - Use the SharedClass class to build the remote connector
 - Change default base model from Model to DataModel
 - Fix DataModel errors not logging correct method names
 - Use the strong-remoting 1.4 resolver API to resolve dynamic remote
methods (relation api)
 - Remove use of fn object for storing remoting meta data
2014-05-19 15:56:26 -07:00
Ritchie Martori 2f21f4ec1e Rework replication test 2014-05-15 17:30:58 -07:00