Every sixth request to persist data slowed UI

[meowofficial]

I'm using redux as a state management tool and sembast to persist redux state and to store initial data. When the redux state is changed I update respective records in sembast in middleware. The user is faced with a stack of cards, the state is changed regarding what button he pressed, and then cards change with animation, so the animation is going concurrently with the data persisting. Every sixth animation lags. It looks like sembast can't be used in the UI thread, but as far as I know, there is no way to use sembast in isolate. Am I doing something wrong?

[meowofficial]

I tried sembast_sqflite and it’s seems to fix the problem.

[alextekartik]

Thanks for the report. I'd like to see what is going wrong though in your initial issue. It seems that writing the file is causing the hanging. The cooperator is not doing its (lame) job. indeed sembast_sqflite won't have this issue:

• write operations are in the background
• there is no (lengthy) compact operation

If you can share your original source code, I would be glad to take a look as it is always a pain to try to reproduce a given case. If you cannot some info on how I could reproduce the issue would help:

• size of db
• operations done that trigger the lags
• animation that could easily show the issue.

Thanks!

[meowofficial]

I can't provide full source code, but I can help to reproduce the issue. I think there is no need in redux, here is the sample of UI:

import 'dart:math';

import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  
  // todo: init database
  
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> with SingleTickerProviderStateMixin {
  Color _currentCardColor;
  Color _nextCardColor;

  AnimationController _animationController;
  Tween<double> _scaleTween;

  void _setNextAndSlideRight() {
    setState(() {
      _nextCardColor = Color.fromRGBO(Random().nextInt(256), Random().nextInt(256), Random().nextInt(256), 1.0);
      _animationController.forward(from: 0.0).whenComplete(() {
        setState(() {
          _currentCardColor = _nextCardColor;
          _nextCardColor = null;
        });
        _animationController.reset();
      });
      
      // todo save some data in database
      
    });
  }

  Widget _buildCard({
    @required Color color,
  }) {
    return Padding(
      padding: EdgeInsets.all(8),
      child: SizedBox.expand(
        child: Container(
          decoration: BoxDecoration(
            color: color,
            borderRadius: BorderRadius.circular(10),
            boxShadow: [
              BoxShadow(
                color: Color.fromRGBO(180, 180, 180, 1.0),
                blurRadius: 5.0,
                spreadRadius: 1.0,
              ),
            ],
          ),
          child: ClipRRect(
            borderRadius: BorderRadius.circular(10),
            child: Center(
              child: CupertinoButton(
                child: Text('Click me'),
                onPressed: _setNextAndSlideRight,
              ),
            ),
          ),
        ),
      ),
    );
  }

  Widget _buildFrontCard() {
    return _SlideRightAnimation(
      controller: _animationController,
      child: _buildCard(
        color: _currentCardColor,
      ),
    );
  }

  Widget _buildBackCard() {
    var backCard = _buildCard(
      color: _nextCardColor,
    );
    return ScaleTransition(
      scale: _scaleTween.animate(_animationController),
      child: backCard,
    );
  }

  @override
  void initState() {
    super.initState();
    _currentCardColor = Color.fromRGBO(Random().nextInt(256), Random().nextInt(256), Random().nextInt(256), 1.0);
    _animationController = AnimationController(
      vsync: this,
      duration: Duration(milliseconds: 300),
    );
    _scaleTween = Tween<double>(
      begin: 0.9,
      end: 1.0,
    );
  }

  @override
  void dispose() {
    _animationController.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return CupertinoPageScaffold(
      navigationBar: CupertinoNavigationBar(
        middle: Text('Title'),
        backgroundColor: Colors.white,
      ),
      child: Container(
        color: CupertinoColors.extraLightBackgroundGray,
        child: SafeArea(
          child: Stack(
            fit: StackFit.expand,
            children: [
              if (_nextCardColor != null) _buildBackCard(),
              _buildFrontCard(),
            ],
          ),
        ),
      ),
    );
  }
}

class _SlideRightAnimation extends AnimatedWidget {
  const _SlideRightAnimation({
    @required this.controller,
    @required this.child,
  }) : super(listenable: controller);

  final Widget child;
  final AnimationController controller;

  @override
  Widget build(BuildContext context) {
    var size = MediaQuery.of(context).size;
    var width = size.width;
    var height = size.height;
    var alpha = pi / 16;
    var dx = (cos(atan(height / width) - alpha) * sqrt(width * width + height * height) - width) / 4;
    return Transform(
      transform: Matrix4.translationValues(
        (dx + width) * controller.value,
        0.0,
        0.0,
      )..rotateZ(alpha * controller.value),
      origin: Offset(width / 2, height / 2),
      child: child,
    );
  }
}

Store format: StoreRef<String, dynamic>('store_name');
In this store I have about 5 records, but I think 2 records will be enough. The biggest one that occupies about 99% of whole database space is initial data which is list of 4000 Maps. The operation that load UI is saving of small Map into second record:
_store.record(_smallMapkey).put(_database, _smallMap);
Database content size is 5mb.

[meowofficial]

Unfortunately sembast_sqflite doesn't fit my needs either due to its 1MB limit per record when I need at least 10-15MB. So I totally depend on resolving the issue with sembast. If you somehow know how to extend this limit, I would appreciate you telling me.

[alextekartik]

Indeed native sqlite has a 1MB cursor limit on Android that cannot be changed. sembast itself is not made for big records neither, just the json encoding of such a big record could hang the UI. Like for images and blob, one solution/recommendation is to use an external file for big records and store a reference in sembast. You could use an isolate for encoding and manipulating such file.

Alternatively instead of having a list of 4000 maps in one record, a better alternative would be having 4000 map records.

This might not fit your current design so maybe sembast is not the best solution here.

[meowofficial]

UI hangs when I save a tiny map into neighbor record. Are you sure that lags appear because of untouched big record?
P.S. If you know better solutions for storing such data could you please tell about them?

[alextekartik]

I need to look at it. My fear is that a compact operation is happening causing the whole file to be written again. This could happen if you are deleting/replacing existing records (the more I know about your usage the better I can try to investigate).

I need to tackle this (important) issue unfortunately it won't be in the short term (summertime, sorry...).

If you cannot store each map as a record, I would likely go got a custom solution, creating a background isolate and manage the data in files in the isolate (which will likely be a pain). Otherwise you could try hive which is known for having good performance.

[meowofficial]

Thanks for help!