数据库读写分离,主从同步实现方法

前言

众所周知,随着用户量的增多,数据库操作往往会成为一个系统的瓶颈所在,而且一般的系统“读”的压力远远大于“写”,因此我们可以通过实现数据库的读写分离来提高系统的性能。

实现思路

通过设置主从数据库实现读写分离,主数据库负责“写操作”,从数据库负责“读操作”,根据压力情况,从数据库可以部署多个提高“读”的速度,借此来提高系统总体的性能。

基础知识

<code class="has-numbering">要实现读写分离,就要解决主从数据库数据同步的问题,在主数据库写入数据后要保证从数据库的数据也要更新。
</code>
  • 1
  • 2

主从数据库同步的实现思路如图:
主从同步

主服务器 master 记录数据库操作日志到 Binary log,从服务器开启 i/o 线程将二进制日志记录的操作同步到 relay log(存在从服务器的缓存中),另外 sql 线程将 relay log 日志记录的操作在从服务器执行。
记住这张图,接下来基于这个图实际设置主从数据库。

主从数据库设置的具体步骤

首先要有两个数据库服务器 master、slave(也可以用一个服务器安装两套数据库环境运行在不同端口,slave 也可以举一反三设置多个),我们穷人就买虚拟云服务器玩玩就行 0.0。以下操作假设你的两台服务器上都已经安装好了 mysql 服务。

1.打开 mysql 数据库配置文件

<code class="hljs perl has-numbering">vim /etc/<span class="hljs-keyword">my</span>.cnf</code>
  • 1

2.在主服务器 master 上配置开启 Binary log,主要是在[mysqld]下面添加:

<code class="hljs lasso has-numbering">server<span class="hljs-attribute">-id</span><span class="hljs-subst">=</span><span class="hljs-number">1</span>
<span class="hljs-keyword">log</span><span class="hljs-attribute">-bin</span><span class="hljs-subst">=</span>master<span class="hljs-attribute">-bin</span>
<span class="hljs-keyword">log</span><span class="hljs-attribute">-bin</span><span class="hljs-attribute">-index</span><span class="hljs-subst">=</span>master<span class="hljs-attribute">-bin</span><span class="hljs-built_in">.</span>index</code>
  • 1
  • 2
  • 3

如图:
master my.cnf

3.重启 mysql 服务

<code class="hljs  has-numbering">service mysql restart</code>
  • 1

ps:重启方式随意

4.检查配置效果,进入主数据库并执行

<code class="hljs  has-numbering">mysql> SHOW MASTER STATUS;</code>
  • 1

可以看到下图表示配置没问题,这里面的 File 名:master-bin.000001 我们接下来在从数据库的配置会使用:

5.配置从服务器的 my.cnf

在[mysqld]节点下面添加:
master status

<code class="hljs lasso has-numbering">server<span class="hljs-attribute">-id</span><span class="hljs-subst">=</span><span class="hljs-number">2</span>
relay<span class="hljs-attribute">-log</span><span class="hljs-attribute">-index</span><span class="hljs-subst">=</span>slave<span class="hljs-attribute">-relay</span><span class="hljs-attribute">-bin</span><span class="hljs-built_in">.</span>index
relay<span class="hljs-attribute">-log</span><span class="hljs-subst">=</span>slave<span class="hljs-attribute">-relay</span><span class="hljs-attribute">-bin</span></code>
  • 1
  • 2
  • 3

这里面的 server-id 一定要和主库的不同,如图:
slave my.cnf
配置完成后同样重启从数据库一下

<code class="hljs  has-numbering">service mysql restart</code>
  • 1

6.接下来配置两个数据库的关联

首先我们先建立一个操作主从同步的数据库用户,切换到主数据库执行:

<code class="hljs oxygene has-numbering">mysql> <span class="hljs-keyword">create</span> user repl;
mysql> GRANT REPLICATION SLAVE <span class="hljs-keyword">ON</span> *.* <span class="hljs-keyword">TO</span> <span class="hljs-string">'repl'</span>@<span class="hljs-string">'从 xxx.xxx.xxx.xx'</span> IDENTIFIED <span class="hljs-keyword">BY</span> <span class="hljs-string">'mysql'</span>;
mysql> flush privileges;</code>
  • 1
  • 2
  • 3

这个配置的含义就是创建了一个数据库用户 repl,密码是 mysql, 在从服务器使用 repl 这个账号和主服务器连接的时候,就赋予其 REPLICATION SLAVE 的权限, *.* 表面这个权限是针对主库的所有表的,其中 xxx 就是从服务器的 ip 地址。
进入从数据库后执行:

<code class="hljs ocaml has-numbering">mysql> change master <span class="hljs-keyword">to</span> master_host=<span class="hljs-string">'主 xxx.xxx.xxx.xx'</span>,master_port=<span class="hljs-number">3306</span>,master_user=<span class="hljs-string">'repl'</span>,master_password=<span class="hljs-string">'mysql'</span>,master_log_file=<span class="hljs-string">'master-bin.000001'</span>,master_log_pos=<span class="hljs-number">0</span>;</code>
  • 1

这里面的 xxx 是主服务器 ip,同时配置端口,repl 代表访问主数据库的用户,上述步骤执行完毕后执行 start slave 启动配置:

<code class="hljs  has-numbering">mysql> start slave;</code>
  • 1

start slave
停止主从同步的命令为:

<code class="hljs vbnet has-numbering">mysql> <span class="hljs-keyword">stop</span> slave;</code>
  • 1

查看状态命令,\G 表示换行查看

<code class="hljs tex has-numbering">mysql> show slave status <span class="hljs-command">\G</span>; </code>
  • 1

可以看到状态如下:
slave status
这里看到从数据库已经在等待主库的消息了,接下来在主库的操作,在从库都会执行了。我们可以主库负责写,从库负责读(不要在从库进行写操作),达到读写分离的效果。

我们可以简单测试:

在主数据库中创建一个新的数据库:

<code class="hljs oxygene has-numbering">mysql> <span class="hljs-keyword">create</span> database testsplit;</code>
  • 1

在从数据库查看数据库:

<code class="hljs  has-numbering">mysql> show databases;</code>
  • 1

可以看到从数据库也有 testsplit 这张表了,这里就不上图了,亲测可用。在主数据库插入数据,从数据库也可以查到。
至此已经实现了数据库主从同步

代码层面实现读写分离

上面我们已经有了两个数据库而且已经实现了主从数据库同步,接下来的问题就是在我们的业务代码里面实现读写分离,假设我们使用的是主流的 ssm 的框架开发的 web 项目,这里面我们需要多个数据源。

<code class="has-numbering">在此之前,我们在项目中一般会使用一个数据库用户远程操作数据库(避免直接使用 root 用户),因此我们需要在主从数据库里面都创建一个用户 mysqluser,赋予其增删改查的权限:
</code>
<code class="hljs oxygene has-numbering">mysql> GRANT <span class="hljs-keyword">select</span>,insert,update,delete <span class="hljs-keyword">ON</span> *.* <span class="hljs-keyword">TO</span> <span class="hljs-string">'mysqluser'</span>@<span class="hljs-string">'%'</span> IDENTIFIED <span class="hljs-keyword">BY</span> <span class="hljs-string">'mysqlpassword'</span> <span class="hljs-keyword">WITH</span> GRANT OPTION;</code>

然后我们的程序里就用 mysqluser 这个用户操作数据库:

1.编写 jdbc.propreties

<code class="hljs avrasm has-numbering"><span class="hljs-preprocessor">#mysql 驱动</span>
jdbc<span class="hljs-preprocessor">.driver</span>=<span class="hljs-keyword">com</span><span class="hljs-preprocessor">.mysql</span><span class="hljs-preprocessor">.jdbc</span><span class="hljs-preprocessor">.Driver</span>
<span class="hljs-preprocessor">#主数据库地址</span>
jdbc<span class="hljs-preprocessor">.master</span><span class="hljs-preprocessor">.url</span>=jdbc:mysql://xxx<span class="hljs-preprocessor">.xxx</span><span class="hljs-preprocessor">.xxx</span><span class="hljs-preprocessor">.xx</span>:<span class="hljs-number">3306</span>/testsplit?useUnicode=true&characterEncoding=utf8
<span class="hljs-preprocessor">#从数据库地址</span>
jdbc<span class="hljs-preprocessor">.slave</span><span class="hljs-preprocessor">.url</span>=jdbc:mysql://xxx<span class="hljs-preprocessor">.xxx</span><span class="hljs-preprocessor">.xxx</span><span class="hljs-preprocessor">.xx</span>:<span class="hljs-number">3306</span>/testsplit?useUnicode=true&characterEncoding=utf8
<span class="hljs-preprocessor">#数据库账号</span>
jdbc<span class="hljs-preprocessor">.username</span>=mysqluser
jdbc<span class="hljs-preprocessor">.password</span>=mysqlpassword</code>

这里我们指定了两个数据库地址,其中的 xxx 分别是我们的主从数据库的 ip 地址,端口都是使用默认的 3306

2.配置数据源

在 spring-dao.xml 中配置数据源(这里就不累赘介绍 spring 的配置了,假设大家都已经配置好运行环境),配置如下:

<code class="hljs xml has-numbering"><span class="hljs-pi"><?xml version="1.0" encoding="UTF-8"?></span>
<span class="hljs-tag"><<span class="hljs-title">beans</span> <span class="hljs-attribute">xmlns</span>=<span class="hljs-value">"http://www.springframework.org/schema/beans"</span>
    <span class="hljs-attribute">xmlns:xsi</span>=<span class="hljs-value">"http://www.w3.org/2001/XMLSchema-instance"</span> <span class="hljs-attribute">xmlns:context</span>=<span class="hljs-value">"http://www.springframework.org/schema/context"</span>
    <span class="hljs-attribute">xsi:schemaLocation</span>=<span class="hljs-value">"http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans.xsd"</span>></span>
    <span class="hljs-comment"><!-- 配置整合 mybatis 过程 --></span>
    <span class="hljs-comment"><!-- 1.配置数据库相关参数 properties 的属性:${url} --></span>
    <span class="hljs-tag"><<span class="hljs-title">context:property-placeholder</span> <span class="hljs-attribute">location</span>=<span class="hljs-value">"classpath:jdbc.properties"</span> /></span>
    <span class="hljs-comment"><!-- 扫描 dao 包下所有使用注解的类型 --></span>
    <span class="hljs-tag"><<span class="hljs-title">context:component-scan</span> <span class="hljs-attribute">base-package</span>=<span class="hljs-value">"c n.xzchain.testsplit.dao"</span> /></span>
    <span class="hljs-comment"><!-- 2.数据库连接池 --></span>
    <span class="hljs-tag"><<span class="hljs-title">bean</span> <span class="hljs-attribute">id</span>=<span class="hljs-value">"abstractDataSource"</span> <span class="hljs-attribute">abstract</span>=<span class="hljs-value">"true"</span> <span class="hljs-attribute">class</span>=<span class="hljs-value">"com.mchange.v2.c3p0.ComboPooledDataSource"</span>
    <span class="hljs-attribute">destroy-method</span>=<span class="hljs-value">"close"</span>></span>
        <span class="hljs-comment"><!-- c3p0 连接池的私有属性 --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"maxPoolSize"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"30"</span> /></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"minPoolSize"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"10"</span> /></span>
        <span class="hljs-comment"><!-- 关闭连接后不自动 commit --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"autoCommitOnClose"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"false"</span> /></span>
        <span class="hljs-comment"><!-- 获取连接超时时间 --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"checkoutTimeout"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"10000"</span> /></span>
        <span class="hljs-comment"><!-- 当获取连接失败重试次数 --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"acquireRetryAttempts"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"2"</span> /></span>
    <span class="hljs-tag"></<span class="hljs-title">bean</span>></span>
    <span class="hljs-comment"><!--主库配置--></span>
    <span class="hljs-tag"><<span class="hljs-title">bean</span> <span class="hljs-attribute">id</span>=<span class="hljs-value">"master"</span> <span class="hljs-attribute">parent</span>=<span class="hljs-value">"abstractDataSource"</span>></span>
        <span class="hljs-comment"><!-- 配置连接池属性 --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"driverClass"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"${jdbc.driver}"</span> /></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"jdbcUrl"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"${jdbc.master.url}"</span> /></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"user"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"${jdbc.username}"</span> /></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"password"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"${jdbc.password}"</span> /></span>
    <span class="hljs-tag"></<span class="hljs-title">bean</span>></span>
    <span class="hljs-comment"><!--从库配置--></span>
    <span class="hljs-tag"><<span class="hljs-title">bean</span> <span class="hljs-attribute">id</span>=<span class="hljs-value">"slave"</span> <span class="hljs-attribute">parent</span>=<span class="hljs-value">"abstractDataSource"</span>></span>
        <span class="hljs-comment"><!-- 配置连接池属性 --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"driverClass"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"${jdbc.driver}"</span> /></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"jdbcUrl"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"${jdbc.slave.url}"</span> /></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"user"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"${jdbc.username}"</span> /></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"password"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"${jdbc.password}"</span> /></span>
    <span class="hljs-tag"></<span class="hljs-title">bean</span>></span>
    <span class="hljs-comment"><!--配置动态数据源,这里的 targetDataSource 就是路由数据源所对应的名称--></span>
    <span class="hljs-tag"><<span class="hljs-title">bean</span> <span class="hljs-attribute">id</span>=<span class="hljs-value">"dataSourceSelector"</span> <span class="hljs-attribute">class</span>=<span class="hljs-value">"cn.xzchain.testsplit.dao.split.DataSourceSelector"</span>></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"targetDataSources"</span>></span>
            <span class="hljs-tag"><<span class="hljs-title">map</span>></span>
                <span class="hljs-tag"><<span class="hljs-title">entry</span> <span class="hljs-attribute">value-ref</span>=<span class="hljs-value">"master"</span> <span class="hljs-attribute">key</span>=<span class="hljs-value">"master"</span>></span><span class="hljs-tag"></<span class="hljs-title">entry</span>></span>
                <span class="hljs-tag"><<span class="hljs-title">entry</span> <span class="hljs-attribute">value-ref</span>=<span class="hljs-value">"slave"</span> <span class="hljs-attribute">key</span>=<span class="hljs-value">"slave"</span>></span><span class="hljs-tag"></<span class="hljs-title">entry</span>></span>
            <span class="hljs-tag"></<span class="hljs-title">map</span>></span>
        <span class="hljs-tag"></<span class="hljs-title">property</span>></span>
    <span class="hljs-tag"></<span class="hljs-title">bean</span>></span>
    <span class="hljs-comment"><!--配置数据源懒加载--></span>
    <span class="hljs-tag"><<span class="hljs-title">bean</span> <span class="hljs-attribute">id</span>=<span class="hljs-value">"dataSource"</span> <span class="hljs-attribute">class</span>=<span class="hljs-value">"org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy"</span>></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"targetDataSource"</span>></span>
            <span class="hljs-tag"><<span class="hljs-title">ref</span> <span class="hljs-attribute">bean</span>=<span class="hljs-value">"dataSourceSelector"</span>></span><span class="hljs-tag"></<span class="hljs-title">ref</span>></span>
        <span class="hljs-tag"></<span class="hljs-title">property</span>></span>
    <span class="hljs-tag"></<span class="hljs-title">bean</span>></span>

    <span class="hljs-comment"><!-- 3.配置 SqlSessionFactory 对象 --></span>
    <span class="hljs-tag"><<span class="hljs-title">bean</span> <span class="hljs-attribute">id</span>=<span class="hljs-value">"sqlSessionFactory"</span> <span class="hljs-attribute">class</span>=<span class="hljs-value">"org.mybatis.spring.SqlSessionFactoryBean"</span>></span>
        <span class="hljs-comment"><!-- 注入数据库连接池 --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"dataSource"</span> <span class="hljs-attribute">ref</span>=<span class="hljs-value">"dataSource"</span> /></span>
        <span class="hljs-comment"><!-- 配置 MyBaties 全局配置文件:mybatis-config.xml --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"configLocation"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"classpath:mybatis-config.xml"</span> /></span>
        <span class="hljs-comment"><!-- 扫描 entity 包 使用别名 --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"typeAliasesPackage"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"cn.xzchain.testsplit.entity"</span> /></span>
        <span class="hljs-comment"><!-- 扫描 sql 配置文件:mapper 需要的 xml 文件 --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"mapperLocations"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"classpath:mapper/*.xml"</span> /></span>
    <span class="hljs-tag"></<span class="hljs-title">bean</span>></span>

    <span class="hljs-comment"><!-- 4.配置扫描 Dao 接口包,动态实现 Dao 接口,注入到 spring 容器中 --></span>
    <span class="hljs-tag"><<span class="hljs-title">bean</span> <span class="hljs-attribute">class</span>=<span class="hljs-value">"org.mybatis.spring.mapper.MapperScannerConfigurer"</span>></span>
        <span class="hljs-comment"><!-- 注入 sqlSessionFactory --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"sqlSessionFactoryBeanName"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"sqlSessionFactory"</span> /></span>
        <span class="hljs-comment"><!-- 给出需要扫描 Dao 接口包 --></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"basePackage"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"cn.xzchain.testsplit.dao"</span> /></span>
    <span class="hljs-tag"></<span class="hljs-title">bean</span>></span>
<span class="hljs-tag"></<span class="hljs-title">beans</span>></span></code>

说明:
首先读取配置文件 jdbc.properties,然后在我们定义了一个基于 c3p0 连接池的父类“抽象”数据源,然后配置了两个具体的数据源 master、slave,继承了 abstractDataSource,这里面就配置了数据库连接的具体属性,然后我们配置了动态数据源,他将决定使用哪个具体的数据源,这里面的关键就是 DataSourceSelector,接下来我们会实现这个 bean。下一步设置了数据源的懒加载,保证在数据源加载的时候其他依赖的 bean 已经加载好了。接着就是常规的配置了,我们的 mybatis 全局配置文件如下

3.mybatis 全局配置文件

<code class="hljs xml has-numbering"><span class="hljs-pi"><?xml version="1.0" encoding="UTF-8" ?></span>
<span class="hljs-doctype"><!DOCTYPE configuration
  PUBLIC "-//mybatis.org//DTD Config 3.0//EN"
  "http://mybatis.org/dtd/mybatis-3-config.dtd"></span>
<span class="hljs-tag"><<span class="hljs-title">configuration</span>></span>
    <span class="hljs-comment"><!-- 配置全局属性 --></span>
    <span class="hljs-tag"><<span class="hljs-title">settings</span>></span>
        <span class="hljs-comment"><!-- 使用 jdbc 的 getGeneratedKeys 获取数据库自增主键值 --></span>
        <span class="hljs-tag"><<span class="hljs-title">setting</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"useGeneratedKeys"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"true"</span> /></span>

        <span class="hljs-comment"><!-- 使用列别名替换列名 默认:true --></span>
        <span class="hljs-tag"><<span class="hljs-title">setting</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"useColumnLabel"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"true"</span> /></span>

        <span class="hljs-comment"><!-- 开启驼峰命名转换:Table{create_time} -> Entity{createTime} --></span>
        <span class="hljs-tag"><<span class="hljs-title">setting</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"mapUnderscoreToCamelCase"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"true"</span> /></span>
        <span class="hljs-comment"><!-- 打印查询语句 --></span>
        <span class="hljs-tag"><<span class="hljs-title">setting</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"logImpl"</span> <span class="hljs-attribute">value</span>=<span class="hljs-value">"STDOUT_LOGGING"</span> /></span>
    <span class="hljs-tag"></<span class="hljs-title">settings</span>></span>
    <span class="hljs-tag"><<span class="hljs-title">plugins</span>></span>
        <span class="hljs-tag"><<span class="hljs-title">plugin</span> <span class="hljs-attribute">interceptor</span>=<span class="hljs-value">"cn.xzchain.testsplit.dao.split.DateSourceSelectInterceptor"</span>></span><span class="hljs-tag"></<span class="hljs-title">plugin</span>></span>
    <span class="hljs-tag"></<span class="hljs-title">plugins</span>></span>
<span class="hljs-tag"></<span class="hljs-title">configuration</span>></span></code>

这里面的关键就是 DateSourceSelectInterceptor 这个拦截器,它会拦截所有的数据库操作,然后分析 sql 语句判断是“读”操作还是“写”操作,我们接下来就来实现上述的 DataSourceSelector 和 DateSourceSelectInterceptor

4.编写 DataSourceSelector

DataSourceSelector 就是我们在 spring-dao.xml 配置的,用于动态配置数据源。代码如下:

<code class="hljs java has-numbering"><span class="hljs-keyword">import</span> org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;

<span class="hljs-javadoc">/**
 *<span class="hljs-javadoctag"> @author</span> lihang
 *<span class="hljs-javadoctag"> @date</span> 2017/12/6.
 *<span class="hljs-javadoctag"> @description</span> 继承了 AbstractRoutingDataSource,动态选择数据源
 */</span>
<span class="hljs-keyword">public</span> <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">DataSourceSelector</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">AbstractRoutingDataSource</span> {</span>

    <span class="hljs-annotation">@Override</span>
    <span class="hljs-keyword">protected</span> Object <span class="hljs-title">determineCurrentLookupKey</span>() {
        <span class="hljs-keyword">return</span> DynamicDataSourceHolder.getDataSourceType();
    }
}</code>

我们只要继承 AbstractRoutingDataSource 并且重写 determineCurrentLookupKey()方法就可以动态配置我们的数据源。
编写 DynamicDataSourceHolder,代码如下:

<code class="hljs java has-numbering"><span class="hljs-javadoc">/**
 *<span class="hljs-javadoctag"> @author</span> lihang
 *<span class="hljs-javadoctag"> @date</span> 2017/12/6.
 *<span class="hljs-javadoctag"> @description</span>
 */</span>
<span class="hljs-keyword">public</span> <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">DynamicDataSourceHolder</span> {</span>

    <span class="hljs-javadoc">/**用来存取 key,ThreadLocal 保证了线程安全*/</span>
    <span class="hljs-keyword">private</span> <span class="hljs-keyword">static</span> ThreadLocal<String> contextHolder = <span class="hljs-keyword">new</span> ThreadLocal<String>();
    <span class="hljs-javadoc">/**主库*/</span>
    <span class="hljs-keyword">public</span> <span class="hljs-keyword">static</span> <span class="hljs-keyword">final</span> String DB_MASTER = <span class="hljs-string">"master"</span>;
    <span class="hljs-javadoc">/**从库*/</span>
    <span class="hljs-keyword">public</span> <span class="hljs-keyword">static</span> <span class="hljs-keyword">final</span> String DB_SLAVE = <span class="hljs-string">"slave"</span>;

    <span class="hljs-javadoc">/**
     * 获取线程的数据源
     *<span class="hljs-javadoctag"> @return</span>
     */</span>
    <span class="hljs-keyword">public</span> <span class="hljs-keyword">static</span> String <span class="hljs-title">getDataSourceType</span>() {
        String db = contextHolder.get();
        <span class="hljs-keyword">if</span> (db == <span class="hljs-keyword">null</span>){
            <span class="hljs-comment">//如果 db 为空则默认使用主库(因为主库支持读和写)</span>
            db = DB_MASTER;
        }
        <span class="hljs-keyword">return</span> db;
    }

    <span class="hljs-javadoc">/**
     * 设置线程的数据源
     *<span class="hljs-javadoctag"> @param</span> s
     */</span>
    <span class="hljs-keyword">public</span> <span class="hljs-keyword">static</span> <span class="hljs-keyword">void</span> <span class="hljs-title">setDataSourceType</span>(String s) {
        contextHolder.set(s);
    }

    <span class="hljs-javadoc">/**
     * 清理连接类型
     */</span>
    <span class="hljs-keyword">public</span> <span class="hljs-keyword">static</span> <span class="hljs-keyword">void</span> <span class="hljs-title">clearDataSource</span>(){
        contextHolder.remove();
    }
}
</code>

这个类决定返回的数据源是 master 还是 slave,这个类的初始化我们就需要借助 DateSourceSelectInterceptor 了,我们拦截所有的数据库操作请求,通过分析 sql 语句来判断是读还是写操作,读操作就给 DynamicDataSourceHolder 设置 slave 源,写操作就给其设置 master 源,代码如下:

<code class="hljs java has-numbering"><span class="hljs-keyword">import</span> org.apache.ibatis.executor.Executor;
<span class="hljs-keyword">import</span> org.apache.ibatis.executor.keygen.SelectKeyGenerator;
<span class="hljs-keyword">import</span> org.apache.ibatis.mapping.BoundSql;
<span class="hljs-keyword">import</span> org.apache.ibatis.mapping.MappedStatement;
<span class="hljs-keyword">import</span> org.apache.ibatis.mapping.SqlCommandType;
<span class="hljs-keyword">import</span> org.apache.ibatis.plugin.*;
<span class="hljs-keyword">import</span> org.apache.ibatis.session.ResultHandler;
<span class="hljs-keyword">import</span> org.apache.ibatis.session.RowBounds;
<span class="hljs-keyword">import</span> org.springframework.transaction.support.TransactionSynchronizationManager;

<span class="hljs-keyword">import</span> java.util.Locale;
<span class="hljs-keyword">import</span> java.util.Properties;

<span class="hljs-javadoc">/**
 *<span class="hljs-javadoctag"> @author</span> lihang
 *<span class="hljs-javadoctag"> @date</span> 2017/12/6.
 *<span class="hljs-javadoctag"> @description</span> 拦截数据库操作,根据 sql 判断是读还是写,选择不同的数据源
 */</span>
<span class="hljs-annotation">@Intercepts</span>({<span class="hljs-annotation">@Signature</span>(type = Executor.class,method = <span class="hljs-string">"update"</span>,args = {MappedStatement.class,Object.class}),
<span class="hljs-annotation">@Signature</span>(type = Executor.class,method = <span class="hljs-string">"query"</span>,args = {MappedStatement.class,Object.class, RowBounds.class, ResultHandler.class})})
<span class="hljs-keyword">public</span> <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">DateSourceSelectInterceptor</span> <span class="hljs-keyword">implements</span> <span class="hljs-title">Interceptor</span>{</span>

    <span class="hljs-javadoc">/**正则匹配 insert、delete、update 操作*/</span>
    <span class="hljs-keyword">private</span> <span class="hljs-keyword">static</span> <span class="hljs-keyword">final</span> String REGEX = <span class="hljs-string">".*insert\\\\u0020.*|.*delete\\\\u0020.*|.*update\\\\u0020.*"</span>;

    <span class="hljs-annotation">@Override</span>
    <span class="hljs-keyword">public</span> Object <span class="hljs-title">intercept</span>(Invocation invocation) <span class="hljs-keyword">throws</span> Throwable {
        <span class="hljs-comment">//判断当前操作是否有事务</span>
        <span class="hljs-keyword">boolean</span> synchonizationActive = TransactionSynchronizationManager.isSynchronizationActive();
        <span class="hljs-comment">//获取执行参数</span>
        Object[] objects = invocation.getArgs();
        MappedStatement ms = (MappedStatement) objects[<span class="hljs-number">0</span>];
        <span class="hljs-comment">//默认设置使用主库</span>
        String lookupKey = DynamicDataSourceHolder.DB_MASTER;;
        <span class="hljs-keyword">if</span> (!synchonizationActive){
            <span class="hljs-comment">//读方法</span>
            <span class="hljs-keyword">if</span> (ms.getSqlCommandType().equals(SqlCommandType.SELECT)){
                <span class="hljs-comment">//selectKey 为自增主键(SELECT LAST_INSERT_ID())方法,使用主库</span>
                <span class="hljs-keyword">if</span> (ms.getId().contains(SelectKeyGenerator.SELECT_KEY_SUFFIX)){
                    lookupKey = DynamicDataSourceHolder.DB_MASTER;
                }<span class="hljs-keyword">else</span> {
                    BoundSql boundSql = ms.getSqlSource().getBoundSql(objects[<span class="hljs-number">1</span>]);
                    String sql = boundSql.getSql().toLowerCase(Locale.CHINA).replace(<span class="hljs-string">"[\\t\\n\\r]"</span>,<span class="hljs-string">" "</span>);
                    <span class="hljs-comment">//如果是 insert、delete、update 操作 使用主库</span>
                    <span class="hljs-keyword">if</span> (sql.matches(REGEX)){
                        lookupKey = DynamicDataSourceHolder.DB_MASTER;
                    }<span class="hljs-keyword">else</span> {
                        <span class="hljs-comment">//使用从库</span>
                        lookupKey = DynamicDataSourceHolder.DB_SLAVE;
                    }
                }
            }
        }<span class="hljs-keyword">else</span> {
            <span class="hljs-comment">//一般使用事务的都是写操作,直接使用主库</span>
            lookupKey = DynamicDataSourceHolder.DB_MASTER;
        }
        <span class="hljs-comment">//设置数据源</span>
        DynamicDataSourceHolder.setDataSourceType(lookupKey);
        <span class="hljs-keyword">return</span> invocation.proceed();
    }

    <span class="hljs-annotation">@Override</span>
    <span class="hljs-keyword">public</span> Object <span class="hljs-title">plugin</span>(Object target) {
        <span class="hljs-keyword">if</span> (target <span class="hljs-keyword">instanceof</span> Executor){
            <span class="hljs-comment">//如果是 Executor(执行增删改查操作),则拦截下来</span>
            <span class="hljs-keyword">return</span> Plugin.wrap(target,<span class="hljs-keyword">this</span>);
        }<span class="hljs-keyword">else</span> {
            <span class="hljs-keyword">return</span> target;
        }
    }

    <span class="hljs-annotation">@Override</span>
    <span class="hljs-keyword">public</span> <span class="hljs-keyword">void</span> <span class="hljs-title">setProperties</span>(Properties properties) {

    }
}</code>

通过这个拦截器,所有的 insert、delete、update 操作设置使用 master 源,select 会使用 slave 源。

接下来就是测试了,我这是生产环境的代码,直接打印日志,小伙伴可以加上日志后测试使用的是哪个数据源,结果和预期一样,这样我们就实现了读写分离~

ps:我们可以配置多个 slave 用于负载均衡,只需要在 spring-dao.xml 中添加 slave1、slave2、slave3……然后修改 dataSourceSelector 这个 bean,

<code class="hljs xml has-numbering"><span class="hljs-tag"><<span class="hljs-title">bean</span> <span class="hljs-attribute">id</span>=<span class="hljs-value">"dataSourceSelector"</span> <span class="hljs-attribute">class</span>=<span class="hljs-value">"cn.xzchain.o2o.dao.split.DataSourceSelector"</span>></span>
        <span class="hljs-tag"><<span class="hljs-title">property</span> <span class="hljs-attribute">name</span>=<span class="hljs-value">"targetDataSources"</span>></span>
            <span class="hljs-tag"><<span class="hljs-title">map</span>></span>
                <span class="hljs-tag"><<span class="hljs-title">entry</span> <span class="hljs-attribute">value-ref</span>=<span class="hljs-value">"master"</span> <span class="hljs-attribute">key</span>=<span class="hljs-value">"master"</span>></span><span class="hljs-tag"></<span class="hljs-title">entry</span>></span>
                <span class="hljs-tag"><<span class="hljs-title">entry</span> <span class="hljs-attribute">value-ref</span>=<span class="hljs-value">"slave1"</span> <span class="hljs-attribute">key</span>=<span class="hljs-value">"slave1"</span>></span><span class="hljs-tag"></<span class="hljs-title">entry</span>></span>
                <span class="hljs-tag"><<span class="hljs-title">entry</span> <span class="hljs-attribute">value-ref</span>=<span class="hljs-value">"slave2"</span> <span class="hljs-attribute">key</span>=<span class="hljs-value">"slave2"</span>></span><span class="hljs-tag"></<span class="hljs-title">entry</span>></span>
                <span class="hljs-tag"><<span class="hljs-title">entry</span> <span class="hljs-attribute">value-ref</span>=<span class="hljs-value">"slave3"</span> <span class="hljs-attribute">key</span>=<span class="hljs-value">"slave3"</span>></span><span class="hljs-tag"></<span class="hljs-title">entry</span>></span>
            <span class="hljs-tag"></<span class="hljs-title">map</span>></span>
        <span class="hljs-tag"></<span class="hljs-title">property</span>></span></code>

在 map 标签中添加 slave1、slave2、slave3……即可,具体的负载均衡策略我们在 DynamicDataSourceHolder、DateSourceSelectInterceptor 中实现即可。

最后整理一下整个流程:
1.项目启动后,在依赖的 bean 加载完成后,我们的数据源通过 LazyConnectionDataSourceProxy 开始加载,他会引用 dataSourceSelector 加载数据源。
2.DataSourceSelector 会选择一个数据源,我们在代码里设置了默认数据源为 master,在初始化的时候我们就默认使用 master 源。
3.在数据库操作执行时,DateSourceSelectInterceptor 拦截器拦截了请求,通过分析 sql 决定使用哪个数据源,“读操作”使用 slave 源,“写操作”使用 master 源。

写在后面

现在很多读写分离中间件已经大大简化了我们的工作,但是自己实现一个小体量的读写分离有助于我们进一步理解数据库读写分离在业务上的实现,呼~


Summer Blog , 版权所有丨如未注明 , 均为原创丨本网站采用BY-NC-SA协议进行授权
转载请注明原文链接:数据库读写分离,主从同步实现方法
喜欢 (0)
发表我的评论
取消评论
表情 贴图 加粗 删除线 居中 斜体 签到

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址