Error creating bean with name 'memcachedClient'...java.lang.OutOfMemoryError

1,Tomcat启动报错以下:java

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'memcachedClient' defined in 

file [/usr/local/apache-tomcat-6.0.37_6500/webapps/trade_service/WEB-INF/classes/META-INF/spring/springContext_memcached.xml]: 

Instantiation of bean failed; nested exception is org.springframework.beans.factory.BeanDefinitionStoreException: Factory method 

[public net.rubyeye.xmemcached.MemcachedClient net.rubyeye.xmemcached.XMemcachedClientBuilder.build() throws java.io.IOException] 

threw exception; nested exception is java.lang.OutOfMemoryError
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod

(ConstructorResolver.java:581)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod

(AbstractAutowireCapableBeanFactory.java:1015)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance

(AbstractAutowireCapableBeanFactory.java:911)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean

(AbstractAutowireCapableBeanFactory.java:485)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean

(AbstractAutowireCapableBeanFactory.java:456)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton

(DefaultSingletonBeanRegistry.java:225)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference

(BeanDefinitionValueResolver.java:322)
        ... 44 more
Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Factory method [public 

net.rubyeye.xmemcached.MemcachedClient net.rubyeye.xmemcached.XMemcachedClientBuilder.build() throws java.io.IOException] threw 

exception; nested exception is java.lang.OutOfMemoryError
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate

(SimpleInstantiationStrategy.java:169)
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod

(ConstructorResolver.java:570)
        ... 53 more
Caused by: java.lang.OutOfMemoryError

,web

2,看到建立memcache clent报错,立刻去memcache服务器查看一下,memcache正常运行,内存也足够剩余。spring

[root@localhost ~]# tail -f /tmp/memcache_error.log 
<287    0x00 0x00 0x00 0x00
<287    0x00 0x00 0x00 0x00
<287    0x00 0x00 0x00 0x00
>287 Writing bin response:
>287   0x81 0x0b 0x00 0x00
>287   0x00 0x00 0x00 0x00
>287   0x00 0x00 0x00 0x06
>287   0x00 0x00 0x00 0x00
>287   0x00 0x00 0x00 0x00
>287   0x00 0x00 0x00 0x00
...
[root@localhost ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          5938       3368       2570          0        195        296
-/+ buffers/cache:       2876       3061
Swap:         7935          0       7935
[root@localhost ~]#
问题在哪里?再看看日志里面有误error或者warning信息:

[root@localhost ~]# cat  /tmp/memcache_error.log  |grep warning
[root@localhost ~]# cat  /tmp/memcache_error.log  |grep error
[root@localhost ~]#
一点错误信息也没有,判断问题不是出在memcache这一块。

3,去check tomcat服务器

[root@localhost logs]# free -m
             total       used       free     shared    buffers     cached
Mem:         18022      16943       1078          0        104        675
-/+ buffers/cache:      16162       1859
Swap:        20031       8012      12019
[root@localhost logs]#
只剩余1个G的内存,去修改一下tomcat的内存配置:

[root@localhost bin]# vim catalina.shsql

#JAVA_OPTS="-Xms2048m -Xmx2048m -Xss1024K -XX:PermSize=256m -XX:MaxPermSize=768m"
# add by wangqi on 20140614
 JAVA_OPTS="-Xms512m -Xmx512m -Xss128K -XX:PermSize=64m -XX:MaxPermSize=256m"
再重启tomcat,看到仍是报同样的错误。


4,回到应用自己去查问题apache

跟开发人员一块儿排查,看到是trade_service启动了过多的多余的事务,把tomcat的内存耗光了,因此关闭掉多余的事务,再启动tomcat,就不会再报错。vim


感慨:就跟大部分web应用卡,页面登陆不上去,相应过慢不少是出在效率低下的sql上面,tomcat启动不起来,内存溢出大部分也是在java代码上面,你们须要写代码或者sql须要谨慎,且写且谨慎。tomcat