Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

These commands take a long time, how to optimize? #60

Open
ljye110 opened this issue Mar 1, 2019 · 2 comments
Open

These commands take a long time, how to optimize? #60

ljye110 opened this issue Mar 1, 2019 · 2 comments
Assignees

Comments

@ljye110
Copy link

ljye110 commented Mar 1, 2019

Commandstats

cmdstat_ttl:calls=8,usec=32,usec_per_call=4.00
cmdstat_scan:calls=6,usec=62741,usec_per_call=10456.83
cmdstat_sadd:calls=89931,usec=670691,usec_per_call=7.46
cmdstat_FT.AGGREGATE:calls=39,usec=2427,usec_per_call=62.23
cmdstat_smembers:calls=6492346,usec=71939185,usec_per_call=11.08
cmdstat_FT.DEL:calls=658,usec=284035,usec_per_call=431.66
cmdstat_select:calls=33,usec=197,usec_per_call=5.97
cmdstat_hgetall:calls=46472001,usec=470146733,usec_per_call=10.12
cmdstat_psync:calls=2,usec=30403,usec_per_call=15201.50
cmdstat_hdel:calls=2454,usec=14804,usec_per_call=6.03
cmdstat_lpush:calls=61462,usec=309028,usec_per_call=5.03
cmdstat_set:calls=67,usec=1282,usec_per_call=19.13
cmdstat_hget:calls=2246056,usec=12474532,usec_per_call=5.55
cmdstat_memory:calls=14,usec=187791,usec_per_call=13413.64
cmdstat_hset:calls=122278,usec=1534612,usec_per_call=12.55
cmdstat_del:calls=29274,usec=190000,usec_per_call=6.49
cmdstat_sismember:calls=192789,usec=1027962,usec_per_call=5.33
cmdstat_FT.ADD:calls=13174,usec=3536154,usec_per_call=268.42
cmdstat_slowlog:calls=24,usec=2865,usec_per_call=119.38
cmdstat_type:calls=8,usec=55,usec_per_call=6.88
cmdstat_sscan:calls=8,usec=441,usec_per_call=55.12
cmdstat_ltrim:calls=5513,usec=71492,usec_per_call=12.97
cmdstat_ping:calls=12670987,usec=11490063,usec_per_call=0.91
cmdstat_zrem:calls=627,usec=13784,usec_per_call=21.98
cmdstat_exists:calls=266233,usec=1245811,usec_per_call=4.68
cmdstat_hmget:calls=1134900,usec=2170039,usec_per_call=1.91
cmdstat_hexists:calls=217658,usec=1042963,usec_per_call=4.79
cmdstat_zrange:calls=278,usec=1887,usec_per_call=6.79
cmdstat_scard:calls=8,usec=65,usec_per_call=8.12
cmdstat_expire:calls=6052,usec=19549,usec_per_call=3.23
cmdstat_srem:calls=63564,usec=162845,usec_per_call=2.56
cmdstat_zadd:calls=7439,usec=228696,usec_per_call=30.74
cmdstat_auth:calls=20567,usec=64597,usec_per_call=3.14
cmdstat_command:calls=12,usec=10349,usec_per_call=862.42
cmdstat_FT.SEARCH:calls=30932,usec=2941357,usec_per_call=95.09
cmdstat_info:calls=39,usec=16068,usec_per_call=412.00
cmdstat_replconf:calls=37577,usec=161327,usec_per_call=4.29
cmdstat_setex:calls=625,usec=14001,usec_per_call=22.40
cmdstat_get:calls=1556628,usec=7468518,usec_per_call=4.80
cmdstat_hmset:calls=74148,usec=2300661,usec_per_call=31.03
cmdstat_lrange:calls=33129,usec=327993,usec_per_call=9.90

cmdstat_FT.AGGREGATE
cmdstat_FT.DEL
cmdstat_FT.ADD
cmdstat_FT.SEARCH
These commands take a long time, how to optimize?

@gkorland
Copy link
Contributor

gkorland commented Mar 5, 2019

@ljye110 can you be more specific? what are you running?
Also https://groups.google.com/forum/#!forum/redisearch might be a better channel for such questions

@ljye110
Copy link
Author

ljye110 commented Mar 8, 2019

    JredisearchClient client = getRediSearchClient();
    Schema.TextField appidField = new Schema.TextField(APPID.toLowerCase(), 1.0, false, true, false);
    Schema.TextField groupField = new Schema.TextField(GROUPID.toLowerCase(), 1.0, false, true, false);

    Schema.TextField classnameField = new Schema.TextField(CLASSNAME.toLowerCase(), 1.0, true, true, false);
    Schema.TextField mpointField = new Schema.TextField(MPOINTCLASS.toLowerCase(), 1.0, true, true, false);
    Schema.TextField ackoperatorField = new Schema.TextField(ACK_OPARATOR.toLowerCase(), 1.0, false, true, false);
    Schema.TextField ruleidField = new Schema.TextField("str_" + RULEID.toLowerCase(), 1.0, false, true, false);
    Schema.TextField kpinoField = new Schema.TextField("str_" + KPINO.toLowerCase(), 1.0, false, true, false);

    Schema sc = new Schema().addField(appidField)
            .addField(groupField) 
            .addSortableNumericField(SEVERITY.toLowerCase()) 
            .addNumericField(KPINO.toLowerCase())
            .addSortableTextField(SEARCHCODE.toLowerCase(), 1.0)  
            .addSortableTextField(MPOINTSEARCHCODE.toLowerCase(), 1.0) 
            .addField(classnameField)
            .addField(mpointField)

            .addSortableNumericField(LAST_OCCURTIME.toLowerCase())
            .addNumericField(DISPATCHSTATUS.toLowerCase())
            .addNumericField(ACK_STATUS.toLowerCase())
            .addField(ackoperatorField)
            .addSortableNumericField(TOTAL.toLowerCase())

            .addField(kpinoField) 
            .addField(ruleidField) 

            .addNumericField("sec_"+OCCURTIME.toLowerCase()) 
            .addNumericField("sec_"+LAST_OCCURTIME.toLowerCase()) 
            .addNumericField("sec_"+ACK_TIME.toLowerCase())   
            .addNumericField("sec_"+UPGRADE_TIME.toLowerCase()) 
            .addTagField("tag_"+SEARCHCODE.toLowerCase())
            .addTagField("tag_"+MPOINTSEARCHCODE.toLowerCase())
            .addTagField("tag_"+TITLE.toLowerCase()) 
            .addTagField("tag_"+TEXT.toLowerCase()) 
            .addTagField("tag_" + HOSTIP.toLowerCase()) 
            .addTagField("tag_" + EVENTID.toLowerCase()); 

    client.dropIndex(FM_ALERT_REDISEARCH_INDEX, true);
    client.createIndex(FM_ALERT_REDISEARCH_INDEX, sc, Client.IndexOptions.defaultOptions());
	
	// addDocument
    Map<String, Object> docMap = new HashMap<>();
    map.put(APPID.toLowerCase(), event.getAppId()); 
    map.put(GROUPID.toLowerCase(), event.getGroupId()); 
    map.put(SEVERITY.toLowerCase(), event.getSeverity()); 
    map.put(KPINO.toLowerCase(), event.getKpiNo()); 
    map.put(SEARCHCODE.toLowerCase(), event.getSearchCode()); 
    map.put(MPOINTSEARCHCODE.toLowerCase(), event.getmPointSearchCode());
    map.put(CLASSNAME.toLowerCase(), StringUtils.defaultIfBlank(event.getClassName(), ""));
    map.put(MPOINTCLASS.toLowerCase(), StringUtils.defaultIfBlank(event.getMpointClass(), "")); 
    map.put(LAST_OCCURTIME.toLowerCase(), alert.getLastOccurTime());
    map.put(DISPATCHSTATUS.toLowerCase(), alert.getDispatchStatus());
    map.put(ACK_STATUS.toLowerCase(), alert.getAckStatus());
    map.put(ACK_OPARATOR.toLowerCase(), alert.getAckOparator());
    map.put(TOTAL.toLowerCase(), alert.getTotal()); 
    map.put("str_" + KPINO.toLowerCase(), "str" + String.valueOf(event.getKpiNo()).replace("-", "x")); 
    map.put("str_" + RULEID.toLowerCase(), "str" + String.valueOf(event.getRuleId()).replace("-", "x")); 
    map.put("sec_" + OCCURTIME.toLowerCase(), DateTimeUtil.Millis2Second(event.getOccurTime()) + ""); 
    map.put("sec_" + LAST_OCCURTIME.toLowerCase(), DateTimeUtil.Millis2Second(alert.getLastOccurTime())); 
    map.put("sec_" + ACK_TIME.toLowerCase(), DateTimeUtil.Millis2Second(alert.getAckTime()) + ""); 
    map.put("sec_" + UPGRADE_TIME.toLowerCase(), DateTimeUtil.Millis2Second(alert.getUpgradeTime())); 
    map.put("tag_" + SEARCHCODE.toLowerCase(), event.getSearchCode()); 
    map.put("tag_" + MPOINTSEARCHCODE.toLowerCase(), event.getmPointSearchCode());
    map.put("tag_" + TITLE.toLowerCase(), event.getTitle()); 
    map.put("tag_" + TEXT.toLowerCase(), event.getText()); 
    map.put("tag_" + HOSTIP.toLowerCase(), StringUtils.defaultIfBlank(event.getHostIP(), "")); 
    map.put("tag_" + EVENTID.toLowerCase(), "tag" + String.valueOf(event.getEventId()).replace("-", "x")); 

	client.addDocument(FaultCommonConst.FM_ALERT_REDISEARCH_INDEX, key, 1.0, docMap, true, false, null);
	
	
	
	// search
	Node n = QueryBuilder.intersect().add(APPID.toLowerCase(), appId)  
                .add(EVENTID.toLowerCase(), Values.eq(eventId)); 
                .add("tag_"+EVENTID.toLowerCase(), Values.tags(eventIdStr)); 

    Query q = new Query(n.toString()).limit(0, 1); 
    q.setLanguage("chinese");

    SearchResult sr = client.search(FaultCommonConst.FM_ALERT_REDISEARCH_INDEX, q);

@gkorland gkorland self-assigned this Mar 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants