首页 > 编程知识 正文

全社会用电量数据查询,游客量数据查询

时间:2023-05-06 07:17:24 阅读:256860 作者:4948

1、场景引入


queryForList默认查询10条每次,最多可设置为10000条,意味着该api最多只能查询10000条数数据
elasticsearchTemplate.queryForList(query, ComparisonPlatformGoodsItemSearchView.class);

类似同步之类的逻辑,es特定的业务索引几百万乃至上千万的数据,显然,仅仅通过elasticsearchTemplate.queryForList()是无法满足的。

2、解决方案: 第一种:通过scroll(类似mysql游标)

官方手册:https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#elasticsearch.scroll

(注意:spring-data-elasticsearch不同版本api名称不一样,4.x.x和3.x.x存在不一样,诸如:searchScrollStart/scrollStart、searchScrollContinue/scrollContinue等)

代码示例:

public void createPurchaseActivityConfigurationGoodsSearchData(PurchaseActivityConfiguration paramBean) { BoolQueryBuilder boolQuery = QueryBuilders.boolQuery() .must(QueryBuilders.matchQuery("brandId", paramBean.getBrandId())) .must(QueryBuilders.matchQuery("comparisonCategoryId", paramBean.getCategoryId())); NativeSearchQuery query = new NativeSearchQueryBuilder().withQuery(boolQuery).build(); List<ComparisonPlatformGoodsItemSearchView> goodsItemViews = new ArrayList<>(); ScrolledPage scroll = elasticsearchTemplate.startScroll(SCROLL_TIME, query, ComparisonPlatformGoodsItemSearchView.class); while (scroll.hasContent()) { goodsItemViews.addAll(scroll.getContent()); scroll = elasticsearchTemplate.continueScroll(scroll.getScrollId(), SCROLL_TIME, ComparisonPlatformGoodsItemSearchView.class); } elasticsearchTemplate.clearScroll(scroll.getScrollId()); if (ObjectUtil.isNotNull(paramBean.getCommissionRatioThreshold())) { purchaseHighSubsidyGoodsSearchRepository.saveAll(covertPurchaseHighSubsidyGoodsSearchView(goodsItemViews)); log.info(">>>>>> createPurchaseHighSubsidyGoodsSearchData success"); } }

 

第二种:通过Stream

版权声明:该文观点仅代表作者本人。处理文章:请发送邮件至 三1五14八八95#扣扣.com 举报,一经查实,本站将立刻删除。