[{"data":1,"prerenderedAt":238},["ShallowReactive",2],{"tag-管理-1":3},{"title":4,"articles":5,"hot_article":27,"month_arch":86,"total":234,"cover":235},"管理 - 独孤影的博客",[6,17],{"id":7,"title":8,"uri":8,"author":9,"tags":10,"created_at":14,"view_count":15,"content":16},39,"结构化数据缓存管理设计","duguying",[11,12,13],"数据库","缓存","管理","2016-07-08 15:55:36",3471,"\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">现代web开发中，在数据库和应用程序服务器之间使用缓存是常见的手段，它有效的减轻了后端数据库服务器的压力。结构化数据库中的数据是按照表、字段这种结构存储的，而作为缓存系统的memcached或是redis-server是以key-value的方式存储数据的。如何将结构化的数据存储到非结构化的缓存系统是一个问题，对于MySQL等结构化数据库我们可以按照以下方式存储。\u003C\u002Fspan>\u003C\u002Fp>\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">首先，对于表中单条数据我们可以按照Bean存储在缓存中，key为“表名+id”，value为Bean对象。\u003C\u002Fspan>\u003C\u002Fp>\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">然后，对于列表查询，我们可以使用“select id from article where status=1 limit 10”这种语句查出article表的id列表 List<Long> 。然后通过id从缓存中逐条加载 Bean ，最终形成我们需要的 List<Bean>。而我们可以将通过sql查出来的List<Long>结构的id数组作为一个缓存对象，其key可以按照某种规则生成，例如“方法名+override_index+参数...”。\u003C\u002Fspan>\u003C\u002Fp>\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">上面说的两种数据对象的存储方式都没有问题，而其中的难点在于如何管理缓存，我们知道，一个列表查询的结果List<Long>在这张表发生增删改的时候都会发生变化，因此，详细来说我们应该在列表的某个字段发生增删改的时候清理掉这个id列表缓存，等下次查询的时候再自动重建新的缓存，以保证缓存的正确性。不过，若是联表查询呢？联表查询“select user.id from article, user where article.userid=user.id and article.status=1”查出的是一个id列表，我们将其缓存假设key为key1，但是一旦article表和user表其中之一发生增删改，那么这个缓存数据应当失效，也就是说，Article在进行 Update()、Save()、Delete() 的时候我们应该清理掉这个key为key1的缓存，同样对于User在进行 \u003Cspan style=\"line-height: 22.5px;\">Update()、Save()、Delete() 的时候我们也应当清理掉 key1 的缓存。在这儿只是列举了两个表联查，并且只是提到了一个联表查询，假如有N个表联表，而且有M个联表查询语句那么我们应该在多少个方法中调用清理这种缓存的方法呢M*N*3，好多好麻烦呀。\u003C\u002Fspan>\u003C\u002Fspan>\u003C\u002Fp>\u003Cp>为了管理list的缓存，定义一个树形结构如下\u003C\u002Fp>\u003Cp>\u003Cimg src=\"\u002F\u002Fduguying.oss-cn-qingdao.aliyuncs.com\u002F2016\u002F7\u002F8\u002Fcache_manage_struct.png\" title=\"\" alt=\"cache_manage_struct.png\"\u002F>\u003C\u002Fp>\u003Cp>t开头的节点表示表名，tA、tB 表示 A 表和 B 表，f开头的表示字段，fAa 表示 A 表 a 字段，fAb 表示 A 表 b 字段，k开头的表示列表的缓存键。这个数据结构表达了我们应该如何清理缓存，比如A表中的a字段发生的改变（增删改），那么我应该清理掉键名为k1, k2, k3的缓存，A表字段b发生增删改时应该清理掉键名为k1, k2的缓存，B表a字段发生增删改时应该清理掉键名为 k1 的缓存。然后将该树形结构拆分为三级，分别用三类的key-value存储起来就好，也不必担心结构过于庞大，上图可以拆为6个key-value：\u003C\u002Fp>\u003Cpre class=\"brush:plain;toolbar:false\">[0]    _init->(tA, tB)\n[1]    tA->(fAa, fAb)\n[1]    tB->(fBa, fBb)\n[2]    fAa->(k1, k2, k3)\n[2]    fAb->(k1, k2)\n[2]    fBa->(k1)\u003C\u002Fpre>\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">\u003Cspan style=\"line-height: 22.5px;\">上图的结构来自于如下三个SQL查询\u003C\u002Fspan>\u003C\u002Fspan>\u003C\u002Fp>\u003Cpre class=\"brush:sql;toolbar:false\">[K1]   select id from A, B  where  A.fAa = ? and A.fAb = ? and B.fBa = ? ...\n[K2]   select id from A     where  A.fAa = ? and A.fAb = ? ...\n[K3]   select id from A     where  A.fAa = ? ...\u003C\u002Fpre>\u003Cp>\u003Cspan style=\"line-height: 22.5px;\">\u003Cspan style=\"line-height: 22.5px;\">\u003C\u002Fspan>\u003C\u002Fspan>\u003Cbr\u002F>\u003C\u002Fp>",{"id":18,"title":19,"uri":19,"author":9,"tags":20,"created_at":24,"view_count":25,"content":26},6,"Windows上的软件包管理器chocolatey",[21,22,23],"chocolatey","windows","软件包管理","2014-08-10 15:55:22",8012,"\u003Cp>软件包管理器相信用过非Windows的人都知道。例如Linux上的apt-get，yum；MacOS上的brew。对于开发者来说，包管理器确实是个好东西，前面所说的包管理器可以安装二进制文件，部分也可以源码编译安装。Windows上的自带的软件管理(添加删除程序)一向被人们称为垃圾。下面就给大家介绍一个第三方实现的包管理器chocolatey。\u003C\u002Fp>\u003Cp>chocolatey官网\u003Ca href=\"http:\u002F\u002Fchocolatey.org\u002F\">http:\u002F\u002Fchocolatey.org\u002F\u003C\u002Fa> ，安装方法：在cmd中执行下面命令(依赖于.NET Framework 4和Powershell)\u003C\u002Fp>\u003Cpre class=\"brush:ps;\">C:>@powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString('https:\u002F\u002Fchocolatey.org\u002Finstall.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\\chocolatey\\bin\u003C\u002Fpre>\u003Cp>之后就可以在cmd中使用一下命令执行安装、卸载、查找软件包了\u003C\u002Fp>\u003Cpre class=\"brush:bat;\">:: 安装notepad2软件\nchoco install notepad2\n:: 或\ncinst notepad2\n:: 卸载notepad2软件\nchoco uninstall notepad2\n:: 或\ncuninst notepad2\n:: 查找软件\nchoco search sublime\u003C\u002Fpre>\u003Cp>与Linux、MacOS上的命令很相似了，不过这个还是无法达到Linux、MacOS上的apt-get,yum,brew那么强悍，因为它只能够安装二进制软件包，并不能做编译安装之类的工作。chocolatey并不够强悍，不过也可以让人们在Windows系统上稍微体验一下命令行安装的感觉了。\u003C\u002Fp>",{"title":28,"list":29},"最热文章",[30,36,42,48,54,60,66,68,74,80],{"id":31,"title":32,"uri":33,"author":9,"created_at":34,"view_count":35},28,"docker容器目录与主机目录同步","\u002Farticle\u002Fdocker容器目录与主机目录同步","2014-12-31 12:34:08",15573,{"id":37,"title":38,"uri":39,"author":9,"created_at":40,"view_count":41},21,"beego中使用fis","\u002Farticle\u002Fbeego中使用fis","2014-12-03 10:47:56",9253,{"id":43,"title":44,"uri":45,"author":9,"created_at":46,"view_count":47},20,"用github登录，oauth开发","\u002Farticle\u002F用github登录，oauth开发","2014-11-23 13:33:37",9115,{"id":49,"title":50,"uri":51,"author":9,"created_at":52,"view_count":53},18,"windows服务注册工具nssm","\u002Farticle\u002Fwindows服务注册工具nssm","2014-10-31 21:34:39",9054,{"id":55,"title":56,"uri":57,"author":9,"created_at":58,"view_count":59},49,"用latex排版中文书籍","\u002Farticle\u002F用latex排版中文书籍","2016-10-31 11:12:02",8645,{"id":61,"title":62,"uri":63,"author":9,"created_at":64,"view_count":65},19,"texlive中文书签乱码","\u002Farticle\u002Ftexlive中文书签乱码","2014-11-15 02:54:01",8466,{"id":18,"title":19,"uri":67,"author":9,"created_at":24,"view_count":25},"\u002Farticle\u002FWindows上的软件包管理器chocolatey",{"id":69,"title":70,"uri":71,"author":9,"created_at":72,"view_count":73},3,"利用开源和免费资源搭建CI环境","\u002Farticle\u002F利用开源和免费资源搭建CI环境","2014-07-04 16:54:58",7740,{"id":75,"title":76,"uri":77,"author":9,"created_at":78,"view_count":79},16,"php-fpm网站目录权限配置","\u002Farticle\u002Fphp-fpm网站目录权限配置","2014-10-18 01:36:10",7276,{"id":81,"title":82,"uri":83,"author":9,"created_at":84,"view_count":85},27,"sublime text 2插件开发，外部修改文件无法重新加载","\u002Farticle\u002Fsublime text 2插件开发，外部修改文件无法重新加载","2014-12-26 20:24:13",7055,{"title":87,"list":88},"按月归档",[89,93,96,99,102,105,108,112,116,119,122,125,129,132,135,139,142,145,148,152,155,158,161,164,167,170,173,176,179,182,185,188,191,194,197,200,203,206,209,212,216,219,222,225,228,231],{"title":90,"uri":91,"view_count":92},"2026年7月","\u002Farchive\u002F2026\u002F7\u002F1","[1]",{"title":94,"uri":95,"view_count":92},"2026年1月","\u002Farchive\u002F2026\u002F1\u002F1",{"title":97,"uri":98,"view_count":92},"2025年4月","\u002Farchive\u002F2025\u002F4\u002F1",{"title":100,"uri":101,"view_count":92},"2024年6月","\u002Farchive\u002F2024\u002F6\u002F1",{"title":103,"uri":104,"view_count":92},"2024年5月","\u002Farchive\u002F2024\u002F5\u002F1",{"title":106,"uri":107,"view_count":92},"2024年4月","\u002Farchive\u002F2024\u002F4\u002F1",{"title":109,"uri":110,"view_count":111},"2023年12月","\u002Farchive\u002F2023\u002F12\u002F1","[6]",{"title":113,"uri":114,"view_count":115},"2023年11月","\u002Farchive\u002F2023\u002F11\u002F1","[2]",{"title":117,"uri":118,"view_count":92},"2023年10月","\u002Farchive\u002F2023\u002F10\u002F1",{"title":120,"uri":121,"view_count":92},"2023年7月","\u002Farchive\u002F2023\u002F7\u002F1",{"title":123,"uri":124,"view_count":115},"2023年6月","\u002Farchive\u002F2023\u002F6\u002F1",{"title":126,"uri":127,"view_count":128},"2023年5月","\u002Farchive\u002F2023\u002F5\u002F1","[3]",{"title":130,"uri":131,"view_count":115},"2023年4月","\u002Farchive\u002F2023\u002F4\u002F1",{"title":133,"uri":134,"view_count":115},"2023年3月","\u002Farchive\u002F2023\u002F3\u002F1",{"title":136,"uri":137,"view_count":138},"2023年2月","\u002Farchive\u002F2023\u002F2\u002F1","[5]",{"title":140,"uri":141,"view_count":92},"2023年1月","\u002Farchive\u002F2023\u002F1\u002F1",{"title":143,"uri":144,"view_count":115},"2022年12月","\u002Farchive\u002F2022\u002F12\u002F1",{"title":146,"uri":147,"view_count":138},"2022年11月","\u002Farchive\u002F2022\u002F11\u002F1",{"title":149,"uri":150,"view_count":151},"2022年10月","\u002Farchive\u002F2022\u002F10\u002F1","[4]",{"title":153,"uri":154,"view_count":111},"2022年9月","\u002Farchive\u002F2022\u002F9\u002F1",{"title":156,"uri":157,"view_count":138},"2022年8月","\u002Farchive\u002F2022\u002F8\u002F1",{"title":159,"uri":160,"view_count":92},"2019年7月","\u002Farchive\u002F2019\u002F7\u002F1",{"title":162,"uri":163,"view_count":92},"2019年2月","\u002Farchive\u002F2019\u002F2\u002F1",{"title":165,"uri":166,"view_count":92},"2018年9月","\u002Farchive\u002F2018\u002F9\u002F1",{"title":168,"uri":169,"view_count":92},"2018年8月","\u002Farchive\u002F2018\u002F8\u002F1",{"title":171,"uri":172,"view_count":92},"2017年11月","\u002Farchive\u002F2017\u002F11\u002F1",{"title":174,"uri":175,"view_count":92},"2017年8月","\u002Farchive\u002F2017\u002F8\u002F1",{"title":177,"uri":178,"view_count":92},"2017年5月","\u002Farchive\u002F2017\u002F5\u002F1",{"title":180,"uri":181,"view_count":92},"2017年4月","\u002Farchive\u002F2017\u002F4\u002F1",{"title":183,"uri":184,"view_count":92},"2016年11月","\u002Farchive\u002F2016\u002F11\u002F1",{"title":186,"uri":187,"view_count":92},"2016年10月","\u002Farchive\u002F2016\u002F10\u002F1",{"title":189,"uri":190,"view_count":115},"2016年9月","\u002Farchive\u002F2016\u002F9\u002F1",{"title":192,"uri":193,"view_count":92},"2016年8月","\u002Farchive\u002F2016\u002F8\u002F1",{"title":195,"uri":196,"view_count":115},"2016年7月","\u002Farchive\u002F2016\u002F7\u002F1",{"title":198,"uri":199,"view_count":138},"2016年6月","\u002Farchive\u002F2016\u002F6\u002F1",{"title":201,"uri":202,"view_count":92},"2015年12月","\u002Farchive\u002F2015\u002F12\u002F1",{"title":204,"uri":205,"view_count":92},"2015年10月","\u002Farchive\u002F2015\u002F10\u002F1",{"title":207,"uri":208,"view_count":115},"2015年2月","\u002Farchive\u002F2015\u002F2\u002F1",{"title":210,"uri":211,"view_count":92},"2015年1月","\u002Farchive\u002F2015\u002F1\u002F1",{"title":213,"uri":214,"view_count":215},"2014年12月","\u002Farchive\u002F2014\u002F12\u002F1","[8]",{"title":217,"uri":218,"view_count":115},"2014年11月","\u002Farchive\u002F2014\u002F11\u002F1",{"title":220,"uri":221,"view_count":138},"2014年10月","\u002Farchive\u002F2014\u002F10\u002F1",{"title":223,"uri":224,"view_count":128},"2014年9月","\u002Farchive\u002F2014\u002F9\u002F1",{"title":226,"uri":227,"view_count":128},"2014年8月","\u002Farchive\u002F2014\u002F8\u002F1",{"title":229,"uri":230,"view_count":128},"2014年7月","\u002Farchive\u002F2014\u002F7\u002F1",{"title":232,"uri":233,"view_count":115},"2013年9月","\u002Farchive\u002F2013\u002F9\u002F1",2,{"url":236,"title":237},"https:\u002F\u002Fimage.duguying.net\u002F2022\u002F09\u002Fij6x8o4s.webp","老家 摄于2020年秋",1790781595405]