SpringBoot压缩json并写入Redis的示例代码
更新时间：2024年08月28日 08:39:02 作者：HBLOG
由于业务需要,存入redis中的缓存数据过大,占用了10+G的内存,内存作为重要资源,需要优化一下大对象缓存,所以我们需要对json进行压缩,本文给大家介绍了SpringBoot如何压缩Json并写入redis,需要的朋友可以参考下
1.为什么需要压缩json？
由于业务需要，存入redis中的缓存数据过大，占用了10+G的内存，内存作为重要资源，需要优化一下大对象缓存，采用gzip压缩存储，可以将 redis 的 kv 对大小缩小大约 7-8 倍，加快存储、读取速度
2.环境搭建
详建redis模块的docker目录
version: '3'
services:
redis:
image: registry.cn-hangzhou.aliyuncs.com/zhengqing/redis:6.0.8
container_name: redis
restart: unless-stopped
command: redis-server /etc/redis/redis.conf --requirepass 123456 --appendonly no
# command: redis-server --requirepass 123456 --appendonly yes
environment:
TZ: Asia/Shanghai
LANG: en_US.UTF-8
volumes:
- "./redis/data:/data"
- "./redis/config/redis.conf:/etc/redis/redis.conf"
ports:
- "6379:6379"
3.代码工程
实验目标
实验存入redis的json数据压缩和解压缩
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>springboot-demo</artifactId>
<groupId>com.et</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>gzip</artifactId>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
<version>2.9.0</version>
</dependency>
</dependencies>
</project>
controller
package com.et.gzip.controller;
import com.et.gzip.model.User;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.HashMap;
import java.util.Map;
@RestController
@Slf4j
public class HelloWorldController {
@Autowired
private RedisTemplate redisTemplateWithJackson;
@PostMapping("/hello")
public User showHelloWorld(@RequestBody User user){
log.info("user:"+ user);
return user;
}
@PostMapping("/redis")
public User redis(@RequestBody User user){
log.info("user:"+ user);
redisTemplateWithJackson.opsForValue().set("user",user);
User redisUser = (User) redisTemplateWithJackson.opsForValue().get("user");
return redisUser;
}
}
redis压缩和解压缩配置
压缩类
package com.et.gzip.config;
import com.et.gzip.model.User;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import lombok.extern.slf4j.Slf4j;
import org.apache.tomcat.util.http.fileupload.IOUtils;
import org.springframework.data.redis.serializer.JdkSerializationRedisSerializer;
import org.springframework.data.redis.serializer.SerializationException;
import sun.misc.BASE64Encoder;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.text.SimpleDateFormat;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
@Slf4j
public class CompressRedis extends JdkSerializationRedisSerializer {
public static final int BUFFER_SIZE = 4096;
private JacksonRedisSerializer<User> jacksonRedisSerializer;
public CompressRedis() {
this.jacksonRedisSerializer = getValueSerializer();
}
@Override
public byte[] serialize(Object graph) throws SerializationException {
if (graph == null) {
return new byte[0];
}
ByteArrayOutputStream bos = null;
GZIPOutputStream gzip = null;
try {
// serialize
byte[] bytes = jacksonRedisSerializer.serialize(graph);
log.info("bytes size{}",bytes.length);
bos = new ByteArrayOutputStream();
gzip = new GZIPOutputStream(bos);
// compress
gzip.write(bytes);
gzip.finish();
byte[] result = bos.toByteArray();
log.info("result size{}",result.length);
//return result;
return new BASE64Encoder().encode(result).getBytes();
} catch (Exception e) {
throw new SerializationException("Gzip Serialization Error", e);
} finally {
IOUtils.closeQuietly(bos);
IOUtils.closeQuietly(gzip);
}
}
@Override
public Object deserialize(byte[] bytes) throws SerializationException {
if (bytes == null || bytes.length == 0) {
return null;
}
ByteArrayOutputStream bos = null;
ByteArrayInputStream bis = null;
GZIPInputStream gzip = null;
try {
bos = new ByteArrayOutputStream();
byte[] compressed = new sun.misc.BASE64Decoder().decodeBuffer( new String(bytes));;
bis = new ByteArrayInputStream(compressed);
gzip = new GZIPInputStream(bis);
byte[] buff = new byte[BUFFER_SIZE];
int n;
// uncompress
while ((n = gzip.read(buff, 0, BUFFER_SIZE)) > 0) {
bos.write(buff, 0, n);
}
//deserialize
Object result = jacksonRedisSerializer.deserialize(bos.toByteArray());
return result;
} catch (Exception e) {
throw new SerializationException("Gzip deserizelie error", e);
} finally {
IOUtils.closeQuietly(bos);
IOUtils.closeQuietly(bis);
IOUtils.closeQuietly(gzip);
}
}
private static JacksonRedisSerializer<User> getValueSerializer() {
JacksonRedisSerializer<User> jackson2JsonRedisSerializer = new JacksonRedisSerializer<>(User.class);
ObjectMapper mapper=new ObjectMapper();
jackson2JsonRedisSerializer.setObjectMapper(mapper);
return jackson2JsonRedisSerializer;
}
}
java序列化
package com.et.gzip.config;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.type.TypeFactory;
import lombok.extern.slf4j.Slf4j;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.SerializationException;
import org.springframework.lang.Nullable;
import org.springframework.util.Assert;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
@Slf4j
public class JacksonRedisSerializer<T> implements RedisSerializer<T> {
public static final Charset DEFAULT_CHARSET;
private final JavaType javaType;
private ObjectMapper objectMapper = new ObjectMapper();
public JacksonRedisSerializer(Class<T> type) {
this.javaType = this.getJavaType(type);
}
public JacksonRedisSerializer(JavaType javaType) {
this.javaType = javaType;
}
public T deserialize(@Nullable byte[] bytes) throws SerializationException {
if (bytes == null || bytes.length == 0) {
return null;
} else {
try {
return this.objectMapper.readValue(bytes, 0, bytes.length, this.javaType);
} catch (Exception var3) {
throw new SerializationException("Could not read JSON: " + var3.getMessage(), var3);
}
}
}
public byte[] serialize(@Nullable Object t) throws SerializationException {
if (t == null) {
return new byte[0];
} else {
try {
return this.objectMapper.writeValueAsBytes(t);
} catch (Exception var3) {
throw new SerializationException("Could not write JSON: " + var3.getMessage(), var3);
}
}
}
public void setObjectMapper(ObjectMapper objectMapper) {
Assert.notNull(objectMapper, "'objectMapper' must not be null");
this.objectMapper = objectMapper;
}
protected JavaType getJavaType(Class<?> clazz) {
return TypeFactory.defaultInstance().constructType(clazz);
}
static {
DEFAULT_CHARSET = StandardCharsets.UTF_8;
}
}
redis序列化
package com.et.gzip.config;
import com.et.gzip.model.User;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
@Configuration
public class RedisWithJacksonConfig {
@Bean(name="redisTemplateWithJackson")
public RedisTemplate<String, User> redisTemplate(LettuceConnectionFactory lettuceConnectionFactory) {
CompressRedis compressRedis = new CompressRedis();
//redisTemplate
RedisTemplate<String, User> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(lettuceConnectionFactory);
RedisSerializer<?> stringSerializer = new StringRedisSerializer();
redisTemplate.setKeySerializer(stringSerializer);
redisTemplate.setValueSerializer(compressRedis);
redisTemplate.setHashKeySerializer(stringSerializer);
redisTemplate.setHashValueSerializer(compressRedis);
redisTemplate.afterPropertiesSet();
return redisTemplate;
}
}
application.yaml
spring:
redis:
host: 127.0.0.1
port: 6379
database: 10
password: 123456
timeout: 10s
lettuce:
pool:
min-idle: 0
max-idle: 8
max-active: 8
max-wait: -1ms
server:
port: 8088
compression:
enabled: true
mime-types: application/json,application/xml,text/html,text/plain,text/css,application/x-javascript
以上只是一些关键代码，所有代码请参见下面代码仓库
代码仓库
4.测试
- 启动spring boot应用
- 用postman访问http://127.0.0.1:8088/redis
可以看到redis里面存储的是gzip压缩的内容
查看控制台日志
2024-08-26 14:37:56.445 INFO 43832 --- [nio-8088-exec-5] com.et.gzip.config.CompressRedis : bytes size371 2024-08-26 14:37:56.445 INFO 43832 --- [nio-8088-exec-5] com.et.gzip.config.CompressRedis : result size58
JSON经过gzip压缩，371-->58, 数据大小减少7-8倍
到此这篇关于SpringBoot压缩json并写入Redis的示例代码的文章就介绍到这了,更多相关SpringBoot压缩json内容请搜索脚本之家以前的文章或继续浏览下面的相关文章希望大家以后多多支持脚本之家！
相关文章
这篇文章主要介绍了详解Spring Boot 配置加载顺序及属性加载顺序,本章内容基于 Spring Boot 2.0 进行详解，感兴趣的朋友跟随脚本之家小编一起看看吧2018-08-08
SpringBoot整合RabbitMQ的5种模式的注解绑定详解
这篇文章主要介绍了SpringBoot整合RabbitMQ的5种模式的注解绑定详解,RabbitMQ 是一个消息中间件,它接收消息并且转发,是"消费-生产者模型"的一个典型的代表,一端往消息队列中不断的写入消息,而另一端则可以读取或者订阅队列中的消息,需要的朋友可以参考下2024-01-01
在软件开发中,随着项目规模的扩大,单个模块的开发方式逐渐转变为多模块开发,这种方式带来了项目管理上的挑战,其中最常见的问题是模块间的依赖管理和版本控制问题,本文就来介绍一下2024-10-10
在数据库里面 ,通常我们会用数字的递进来描述状态等信息 , 但是如果想进行更复杂的操作 , 就有必要对二进制有一定理解了。本文就来趣味性的探讨一下 , 如何通过更少的空间描述更多的信息2022-09-09
Spring Cloud Gateway远程命令执行漏洞分析(CVE-2022-22947)
使用Spring Cloud Gateway的应用程序在Actuator端点启用、公开和不安全的情况下容易受到代码注入的攻击，攻击者可以恶意创建允许在远程主机上执行任意远程执行的请求，这篇文章主要介绍了Spring Cloud Gateway远程命令执行漏洞（CVE-2022-22947）,需要的朋友可以参考下2023-03-03
这篇文章主要介绍了Java 文件传输助手的实现(单机版)，文中通过示例代码介绍的非常详细，对大家的学习或者工作具有一定的参考学习价值，需要的朋友们下面随着小编来一起学习学习吧2020-05-05
这篇文章主要介绍了spring boot中的properties参数配置,需要的朋友可以参考下2017-09-09
SpringBoot中Elasticsearch的连接配置原理与使用详解
Elasticsearch是一种开源的分布式搜索和数据分析引擎,它可用于全文搜索、结构化搜索、分析等应用场景,本文主要介绍了SpringBoot中Elasticsearch的连接配置原理与使用详解,感兴趣的可以了解一下2023-09-09
这篇文章主要介绍了Java中父类和子类之间的转换操作,结合实例形式分析了Java中父类和子类之间的转换相关原理、操作技巧与使用注意事项,需要的朋友可以参考下2020-05-05
这篇文章主要介绍了SpringMVC Hibernate与JPA，在正式进入Hibernate的高级应用之前，需要了解声明是数据模型与领域模型，这两个概念将会帮助我们更好的理解实体对象的关联关系映射2022-12-12
最新评论