[{"data":1,"prerenderedAt":230},["ShallowReactive",2],{"article-结构化数据缓存管理设计":3},{"title":4,"article":5,"hot_article":16,"month_arch":79,"cover":227},"结构化数据缓存管理设计 - 独孤影的博客",{"id":6,"title":7,"uri":7,"author":8,"tags":9,"created_at":13,"view_count":14,"content":15},39,"结构化数据缓存管理设计","duguying",[10,11,12],"数据库","缓存","管理","2016-07-08 15:55:36",3471,"\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">现代web开发中，在数据库和应用程序服务器之间使用缓存是常见的手段，它有效的减轻了后端数据库服务器的压力。结构化数据库中的数据是按照表、字段这种结构存储的，而作为缓存系统的memcached或是redis-server是以key-value的方式存储数据的。如何将结构化的数据存储到非结构化的缓存系统是一个问题，对于MySQL等结构化数据库我们可以按照以下方式存储。\u003C\u002Fspan>\u003C\u002Fp>\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">首先，对于表中单条数据我们可以按照Bean存储在缓存中，key为“表名+id”，value为Bean对象。\u003C\u002Fspan>\u003C\u002Fp>\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">然后，对于列表查询，我们可以使用“select id from article where status=1 limit 10”这种语句查出article表的id列表 List<Long> 。然后通过id从缓存中逐条加载 Bean ，最终形成我们需要的 List<Bean>。而我们可以将通过sql查出来的List<Long>结构的id数组作为一个缓存对象，其key可以按照某种规则生成，例如“方法名+override_index+参数...”。\u003C\u002Fspan>\u003C\u002Fp>\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">上面说的两种数据对象的存储方式都没有问题，而其中的难点在于如何管理缓存，我们知道，一个列表查询的结果List<Long>在这张表发生增删改的时候都会发生变化，因此，详细来说我们应该在列表的某个字段发生增删改的时候清理掉这个id列表缓存，等下次查询的时候再自动重建新的缓存，以保证缓存的正确性。不过，若是联表查询呢？联表查询“select user.id from article, user where article.userid=user.id and article.status=1”查出的是一个id列表，我们将其缓存假设key为key1，但是一旦article表和user表其中之一发生增删改，那么这个缓存数据应当失效，也就是说，Article在进行 Update()、Save()、Delete() 的时候我们应该清理掉这个key为key1的缓存，同样对于User在进行 \u003Cspan style=\"line-height: 22.5px;\">Update()、Save()、Delete() 的时候我们也应当清理掉 key1 的缓存。在这儿只是列举了两个表联查，并且只是提到了一个联表查询，假如有N个表联表，而且有M个联表查询语句那么我们应该在多少个方法中调用清理这种缓存的方法呢M*N*3，好多好麻烦呀。\u003C\u002Fspan>\u003C\u002Fspan>\u003C\u002Fp>\u003Cp>为了管理list的缓存，定义一个树形结构如下\u003C\u002Fp>\u003Cp>\u003Cimg src=\"\u002F\u002Fduguying.oss-cn-qingdao.aliyuncs.com\u002F2016\u002F7\u002F8\u002Fcache_manage_struct.png\" title=\"\" alt=\"cache_manage_struct.png\"\u002F>\u003C\u002Fp>\u003Cp>t开头的节点表示表名，tA、tB 表示 A 表和 B 表，f开头的表示字段，fAa 表示 A 表 a 字段，fAb 表示 A 表 b 字段，k开头的表示列表的缓存键。这个数据结构表达了我们应该如何清理缓存，比如A表中的a字段发生的改变（增删改），那么我应该清理掉键名为k1, k2, k3的缓存，A表字段b发生增删改时应该清理掉键名为k1, k2的缓存，B表a字段发生增删改时应该清理掉键名为 k1 的缓存。然后将该树形结构拆分为三级，分别用三类的key-value存储起来就好，也不必担心结构过于庞大，上图可以拆为6个key-value：\u003C\u002Fp>\u003Cpre class=\"brush:plain;toolbar:false\">[0]    _init->(tA, tB)\n[1]    tA->(fAa, fAb)\n[1]    tB->(fBa, fBb)\n[2]    fAa->(k1, k2, k3)\n[2]    fAb->(k1, k2)\n[2]    fBa->(k1)\u003C\u002Fpre>\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">\u003Cspan style=\"line-height: 22.5px;\">上图的结构来自于如下三个SQL查询\u003C\u002Fspan>\u003C\u002Fspan>\u003C\u002Fp>\u003Cpre class=\"brush:sql;toolbar:false\">[K1]   select id from A, B  where  A.fAa = ? and A.fAb = ? and B.fBa = ? ...\n[K2]   select id from A     where  A.fAa = ? and A.fAb = ? ...\n[K3]   select id from A     where  A.fAa = ? ...\u003C\u002Fpre>\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">\u003Cspan style=\"line-height: 22.5px;\">\u003C\u002Fspan>\u003C\u002Fspan>\u003Cbr\u002F>\u003C\u002Fp>",{"title":17,"list":18},"最热文章",[19,25,31,37,43,49,55,61,67,73],{"id":20,"title":21,"uri":22,"author":8,"created_at":23,"view_count":24},28,"docker容器目录与主机目录同步","\u002Farticle\u002Fdocker容器目录与主机目录同步","2014-12-31 12:34:08",15573,{"id":26,"title":27,"uri":28,"author":8,"created_at":29,"view_count":30},21,"beego中使用fis","\u002Farticle\u002Fbeego中使用fis","2014-12-03 10:47:56",9253,{"id":32,"title":33,"uri":34,"author":8,"created_at":35,"view_count":36},20,"用github登录，oauth开发","\u002Farticle\u002F用github登录，oauth开发","2014-11-23 13:33:37",9115,{"id":38,"title":39,"uri":40,"author":8,"created_at":41,"view_count":42},18,"windows服务注册工具nssm","\u002Farticle\u002Fwindows服务注册工具nssm","2014-10-31 21:34:39",9054,{"id":44,"title":45,"uri":46,"author":8,"created_at":47,"view_count":48},49,"用latex排版中文书籍","\u002Farticle\u002F用latex排版中文书籍","2016-10-31 11:12:02",8645,{"id":50,"title":51,"uri":52,"author":8,"created_at":53,"view_count":54},19,"texlive中文书签乱码","\u002Farticle\u002Ftexlive中文书签乱码","2014-11-15 02:54:01",8466,{"id":56,"title":57,"uri":58,"author":8,"created_at":59,"view_count":60},6,"Windows上的软件包管理器chocolatey","\u002Farticle\u002FWindows上的软件包管理器chocolatey","2014-08-10 15:55:22",8012,{"id":62,"title":63,"uri":64,"author":8,"created_at":65,"view_count":66},3,"利用开源和免费资源搭建CI环境","\u002Farticle\u002F利用开源和免费资源搭建CI环境","2014-07-04 16:54:58",7740,{"id":68,"title":69,"uri":70,"author":8,"created_at":71,"view_count":72},16,"php-fpm网站目录权限配置","\u002Farticle\u002Fphp-fpm网站目录权限配置","2014-10-18 01:36:10",7276,{"id":74,"title":75,"uri":76,"author":8,"created_at":77,"view_count":78},27,"sublime text 2插件开发，外部修改文件无法重新加载","\u002Farticle\u002Fsublime text 2插件开发，外部修改文件无法重新加载","2014-12-26 20:24:13",7055,{"title":80,"list":81},"按月归档",[82,86,89,92,95,98,101,105,109,112,115,118,122,125,128,132,135,138,141,145,148,151,154,157,160,163,166,169,172,175,178,181,184,187,190,193,196,199,202,205,209,212,215,218,221,224],{"title":83,"uri":84,"view_count":85},"2026年7月","\u002Farchive\u002F2026\u002F7\u002F1","[1]",{"title":87,"uri":88,"view_count":85},"2026年1月","\u002Farchive\u002F2026\u002F1\u002F1",{"title":90,"uri":91,"view_count":85},"2025年4月","\u002Farchive\u002F2025\u002F4\u002F1",{"title":93,"uri":94,"view_count":85},"2024年6月","\u002Farchive\u002F2024\u002F6\u002F1",{"title":96,"uri":97,"view_count":85},"2024年5月","\u002Farchive\u002F2024\u002F5\u002F1",{"title":99,"uri":100,"view_count":85},"2024年4月","\u002Farchive\u002F2024\u002F4\u002F1",{"title":102,"uri":103,"view_count":104},"2023年12月","\u002Farchive\u002F2023\u002F12\u002F1","[6]",{"title":106,"uri":107,"view_count":108},"2023年11月","\u002Farchive\u002F2023\u002F11\u002F1","[2]",{"title":110,"uri":111,"view_count":85},"2023年10月","\u002Farchive\u002F2023\u002F10\u002F1",{"title":113,"uri":114,"view_count":85},"2023年7月","\u002Farchive\u002F2023\u002F7\u002F1",{"title":116,"uri":117,"view_count":108},"2023年6月","\u002Farchive\u002F2023\u002F6\u002F1",{"title":119,"uri":120,"view_count":121},"2023年5月","\u002Farchive\u002F2023\u002F5\u002F1","[3]",{"title":123,"uri":124,"view_count":108},"2023年4月","\u002Farchive\u002F2023\u002F4\u002F1",{"title":126,"uri":127,"view_count":108},"2023年3月","\u002Farchive\u002F2023\u002F3\u002F1",{"title":129,"uri":130,"view_count":131},"2023年2月","\u002Farchive\u002F2023\u002F2\u002F1","[5]",{"title":133,"uri":134,"view_count":85},"2023年1月","\u002Farchive\u002F2023\u002F1\u002F1",{"title":136,"uri":137,"view_count":108},"2022年12月","\u002Farchive\u002F2022\u002F12\u002F1",{"title":139,"uri":140,"view_count":131},"2022年11月","\u002Farchive\u002F2022\u002F11\u002F1",{"title":142,"uri":143,"view_count":144},"2022年10月","\u002Farchive\u002F2022\u002F10\u002F1","[4]",{"title":146,"uri":147,"view_count":104},"2022年9月","\u002Farchive\u002F2022\u002F9\u002F1",{"title":149,"uri":150,"view_count":131},"2022年8月","\u002Farchive\u002F2022\u002F8\u002F1",{"title":152,"uri":153,"view_count":85},"2019年7月","\u002Farchive\u002F2019\u002F7\u002F1",{"title":155,"uri":156,"view_count":85},"2019年2月","\u002Farchive\u002F2019\u002F2\u002F1",{"title":158,"uri":159,"view_count":85},"2018年9月","\u002Farchive\u002F2018\u002F9\u002F1",{"title":161,"uri":162,"view_count":85},"2018年8月","\u002Farchive\u002F2018\u002F8\u002F1",{"title":164,"uri":165,"view_count":85},"2017年11月","\u002Farchive\u002F2017\u002F11\u002F1",{"title":167,"uri":168,"view_count":85},"2017年8月","\u002Farchive\u002F2017\u002F8\u002F1",{"title":170,"uri":171,"view_count":85},"2017年5月","\u002Farchive\u002F2017\u002F5\u002F1",{"title":173,"uri":174,"view_count":85},"2017年4月","\u002Farchive\u002F2017\u002F4\u002F1",{"title":176,"uri":177,"view_count":85},"2016年11月","\u002Farchive\u002F2016\u002F11\u002F1",{"title":179,"uri":180,"view_count":85},"2016年10月","\u002Farchive\u002F2016\u002F10\u002F1",{"title":182,"uri":183,"view_count":108},"2016年9月","\u002Farchive\u002F2016\u002F9\u002F1",{"title":185,"uri":186,"view_count":85},"2016年8月","\u002Farchive\u002F2016\u002F8\u002F1",{"title":188,"uri":189,"view_count":108},"2016年7月","\u002Farchive\u002F2016\u002F7\u002F1",{"title":191,"uri":192,"view_count":131},"2016年6月","\u002Farchive\u002F2016\u002F6\u002F1",{"title":194,"uri":195,"view_count":85},"2015年12月","\u002Farchive\u002F2015\u002F12\u002F1",{"title":197,"uri":198,"view_count":85},"2015年10月","\u002Farchive\u002F2015\u002F10\u002F1",{"title":200,"uri":201,"view_count":108},"2015年2月","\u002Farchive\u002F2015\u002F2\u002F1",{"title":203,"uri":204,"view_count":85},"2015年1月","\u002Farchive\u002F2015\u002F1\u002F1",{"title":206,"uri":207,"view_count":208},"2014年12月","\u002Farchive\u002F2014\u002F12\u002F1","[8]",{"title":210,"uri":211,"view_count":108},"2014年11月","\u002Farchive\u002F2014\u002F11\u002F1",{"title":213,"uri":214,"view_count":131},"2014年10月","\u002Farchive\u002F2014\u002F10\u002F1",{"title":216,"uri":217,"view_count":121},"2014年9月","\u002Farchive\u002F2014\u002F9\u002F1",{"title":219,"uri":220,"view_count":121},"2014年8月","\u002Farchive\u002F2014\u002F8\u002F1",{"title":222,"uri":223,"view_count":121},"2014年7月","\u002Farchive\u002F2014\u002F7\u002F1",{"title":225,"uri":226,"view_count":108},"2013年9月","\u002Farchive\u002F2013\u002F9\u002F1",{"url":228,"title":229},"https:\u002F\u002Fimage.duguying.net\u002F2022\u002F09\u002Fij6x8o4s.webp","老家 摄于2020年秋",1790781595253]