标签：old far prometheus scrape manager too go 20
背景描述：aws中的prometheus实例状态检查出现异常，重启实例后实例正常，此时发现grafana中主机为空，经查看日志发现如下：
level=warn ts=2021-08-16T03:20:04.960Z caller=scrape.go:1507 component="scrape manager" scrape_pool=mtail target=http://18.167.146.20:3903/metrics msg="Error on ingesting samples that are too old or are too far into the future" num_dropped=78 level=warn ts=2021-08-16T03:20:04.961Z caller=scrape.go:1203 component="scrape manager" scrape_pool=mtail target=http://18.167.146.20:3903/metrics msg="Appending scrape report failed" err="out of bounds" level=warn ts=2021-08-16T03:20:06.868Z caller=scrape.go:1507 component="scrape manager" scrape_pool=node target=http://18.166.55.107:9100/metrics msg="Error on ingesting samples that are too old or are too far into the future" num_dropped=491 level=warn ts=2021-08-16T03:20:06.868Z caller=scrape.go:1203 component="scrape manager" scrape_pool=node target=http://18.166.55.107:9100/metrics msg="Appending scrape report failed" err="out of bounds" level=warn ts=2021-08-16T03:20:08.140Z caller=scrape.go:1507 component="scrape manager" scrape_pool=mysql-mater-slave-luxe target=http://18.166.83.165:9104/metrics msg="Error on ingesting samples that are too old or are too far into the future" num_dropped=959 level=warn ts=2021-08-16T03:20:08.140Z caller=scrape.go:1203 component="scrape manager" scrape_pool=mysql-mater-slave-luxe target=http://18.166.83.165:9104/metrics msg="Appending scrape report failed" err="out of bounds" level=warn ts=2021-08-16T03:20:09.037Z caller=scrape.go:1507 component="scrape manager" scrape_pool=mysql-mater-slave-luxe target=http://16.162.107.109:9104/metrics msg="Error on ingesting samples that are too old or are too far into the future" num_dropped=941 level=warn ts=2021-08-16T03:20:09.037Z caller=scrape.go:1203 component="scrape manager" scrape_pool=mysql-mater-slave-luxe target=http://16.162.107.109:9104/metrics msg="Appending scrape report failed" err="out of bounds" level=warn ts=2021-08-16T03:20:09.960Z caller=scrape.go:1507 component="scrape manager" scrape_pool=mtail target=http://18.167.146.20:3903/metrics msg="Error on ingesting samples that are too old or are too far into the future" num_dropped=78 level=warn ts=2021-08-16T03:20:09.960Z caller=scrape.go:1203 component="scrape manager" scrape_pool=mtail target=http://18.167.146.20:3903/metrics msg="Appending scrape report failed" err="out of bounds" level=warn ts=2021-08-16T03:20:10.168Z caller=scrape.go:1507 component="scrape manager" scrape_pool=prometheus target=http://18.166.55.107:9090/metrics msg="Error on ingesting samples that are too old or are too far into the future" num_dropped=558 level=warn ts=2021-08-16T03:20:10.168Z caller=scrape.go:1203 component="scrape manager" scrape_pool=prometheus target=http://18.166.55.107:9090/metrics msg="Appending scrape report failed" err="out of bounds"
解决方法：
备份prometheus数据库，新建data目录，重启prometheus和grafana服务 [root@prometheus /opt/prometheus]# mv data/ data_bak [root@prometheus /opt/prometheus]# mkdir -pv data && chown -R prometheus.prometheus data [root@prometheus /opt/prometheus]# systemctl restart prometheus [root@prometheus /opt/prometheus]# systemctl restart grafana-server
web界面访问prometheus正常，grafana访问正常
标签：old,far,prometheus,scrape,manager,too,go,20 来源： https://www.cnblogs.com/alisapine/p/15149435.html
本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享； 2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关； 3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关； 4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除； 5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。