怎么以java代码的方式,直接将csv或者json导入到mongodb里面。

现在有个问题,就是我需要别人传给我数据,而这个数据量也是挺大的,现在实现的逻辑是提供个接口,不断轮询的去调,我这边是mongodbTemplate不断的去保存。

但我总感觉,大数据量的情况下,这种方式好low,查询了许多资料,得知mongodb是支持以csv或json格式直接上传到库的,但都是命令行的形式。

所以求助大家,怎么以java代码的方式,直接将csv或者json导入到mongodb里面。

另外一点就是,mongodb的import不是java读取一行然后解析类型在处理成对象,再去保存,那样和调接口没啥区别,还不如直接调接口,我想的是直接导入csv文件,利用的是mongodb的特性,只要按照格式准备数据就可以了,所以各位大神帮忙看看?万分感谢

1个回答

不懂这个,你说是命令行运行mongodb,那就用java调用命令行啊 Rutime

weixin_43957818
你,别闹我有药i 现在已知的是mongodb的命令行mongoimport是可以导入csv或者json文件,直接导入进collection的,我想通过java代码来实现
大约一年之前 回复
Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
django如何在model导入已经存在的mongo数据库?
我的表在mongo数据库中且结构和内容都是已经完成的,如何在django的某个app里的model中导入mongo里的这张表呢,我搜到了很多关于mysql的,但是没找到mongo的,init里面添加的东西我试了半天pymongo和mongoengine好像都不对,有没有大佬解答一下啊。。。
100万条数据存入mongo,通过java代码实现,要求时间在5秒之内,或者越快越好
通过java代码将oracle中的100万条数据查询出来,然后直接插入mongo数据库中,要求时间越短越好。我从oracle中将100万条数据查询出来花费了3秒左右。希望存入mongo的时间在5秒以内。请问各位大神有没有好的办法,谢谢。
mongo java web command
如何在web端输入mongo命令执行mongo命令,后台是java
怎么在java中实现类似show dbs 这样的mongo语句啊?
怎么在java中实现类似show dbs 这样的mongo语句啊?是用db.eval这个方法吗?具体怎么用啊,求指教啊
求解下 我用java链接mongo有错误
SEVERE: Expecting a single StandAlone, but found more than one. Removing 192.168.0.81:27017 from client view of cluster. Aug 11, 2017 4:43:39 PM com.mongodb.MultiServerCluster handleStandAloneChanged SEVERE: Expecting a single StandAlone, but found more than one. Removing 192.168.0.82:27017 from client view of cluster. Exception in thread "Thread-5" com.mongodb.MongoTimeoutException: Timed out after 10000 ms while waiting to connect. Client view of cluster state is {type=Unknown, servers=[] at com.mongodb.BaseCluster.getDescription(BaseCluster.java:128) at com.mongodb.DBTCPConnector.getClusterDescription(DBTCPConnector.java:396) at com.mongodb.DBTCPConnector.getType(DBTCPConnector.java:569) at com.mongodb.DBTCPConnector.isMongosConnection(DBTCPConnector.java:370) at com.mongodb.Mongo.isMongosConnection(Mongo.java:645) at com.mongodb.DBCollection.findOne(DBCollection.java:865) at com.mongodb.DBCollection.findOne(DBCollection.java:843) at com.mongodb.DBCollection.findOne(DBCollection.java:789) at com.gzace.buzz.up.SendThreadTele.isSysUpgrade(SendThreadTele.java:171) at com.gzace.buzz.up.SendThreadTele.run(SendThreadTele.java:34) 启动jar包时出现这种错误 以前是能正常运行的 而到客户端时 mongo显示 2017-07-14T20:08:03.890+0800 [initandlisten] 2017-07-14T20:08:03.890+0800 [initandlisten] ** WARNING: mongod started without --replSet yet 1 documents are present in local.system.replset 2017-07-14T20:08:03.890+0800 [initandlisten] ** Restart with --replSet unless you are doing maintenance and no other clients are connected. 2017-07-14T20:08:03.890+0800 [initandlisten] ** The TTL collection monitor will not start because of this. 2017-07-14T20:08:03.890+0800 [initandlisten] ** For more info see http://dochub.mongodb.org/core/ttlcollections 2017-07-14T20:08:03.890+0800 [initandlisten] 请问是有什么问题呢
mongo3.0版本数据恢复到mongo3.6版本,提示没有restore索引
1、问题描述 最近部署了一套升级版的mongodb(3.6.5),想把mongo3.0版本备份的数据恢复到最新版本,但是提示没有恢复索引,该如何解决 2、截图 ![图片说明](https://img-ask.csdn.net/upload/201911/05/1572923410_365657.png) 提示没有索引restore,该如何解决恢复索引问题
请教一下mongo数据查询问题
mongo刚入门,想请教一下查询mongo数据后得到的结集有没有方法将两个DBcursor组合到一起进行分页操作,或者说我要查多个mongo集合,需要将这数据放在一起进行分页
php代码中实现 mongo的$geoWithin的问题。求助大神帮忙解决,谢谢
工作中遇到一个问题,现在一个终端商店A的坐标,要看该商店是不是在给定的坐标范围中,给定的坐标范围是有n(n>=1)个圈,而每个圈是有至少3个点围起来的,不同圈之间没有交集,现在想要的结果是,该终端在这n个圈中。 这里使用了mongo的geoWithin。并且,没有使用tp框架。还有就是网上的全是命令行格式的,php代码格式的几乎没有。我现在感觉可能就是格式的问题。有大牛帮忙看一下,谢谢。 $new_lbs_info_arr = json_decode($one_lbsinfo['lbs_info']);//获取的是这n个圈的数组格式; //这一步就是我最困惑的地方 $condition = array('lbs'=>array('$geometry'=>array('$geoWithin'=>array('$polygon'=>$new_lbs_info_arr)))); $is_within = $collection_user_lbs->find($condition);//查询是否在圈内 var_dump($is_within);//结果却是“object(MongoCursor)#12 (0) {}”这种格式的!
PHP连接mongo集群方式
你们谁用的是PHP连接mongo集群方式的 我现在出现一个问题是, new MongoClient(连接单个实例数据库F5刷新浏览器数据显示没有任何问题) 可是new MongoClient(我连接多个IP集群方式,浏览器刷新数据的时候有时候显示数据 有时候没有数据显示。) 这是为什么啊 求解答。。非常非常非常非常感谢感谢。
mongodb 读取不到数据
我在Swarm集群中部署了一个mongo的docker ,然后挂载了一个数据卷 -v data/db:/data/db,然后,将这个数据卷copy到k8s集群中的一个节点上,然后在k8s中部署一个mongo的docker, 也挂载这个卷,但是这个mongo启动之后,之前的数据库都不存在了 ,我查询了导过来的数据卷大小和之前的数据卷大小是一样的, 有没有知道这个问题的原因的,请告知小弟一下
java.lang.TypeNotPresentException: Type org.springframework.jdbc.CannotGetJdbcConnectionException not present
## # 使用springboot-2.0.1.RELEASE整合mongodb-3.4.23版本时报如下错误: ``` java.lang.TypeNotPresentException: Type org.springframework.jdbc.CannotGetJdbcConnectionException not present at sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(CoreReflectionFactory.java:117) at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:125) at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49) at sun.reflect.generics.visitor.Reifier.reifyTypeArguments(Reifier.java:68) at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:138) at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49) at sun.reflect.generics.repository.ClassRepository.getSuperclass(ClassRepository.java:90) at java.lang.Class.getGenericSuperclass(Class.java:777) at org.springframework.core.ResolvableType.getSuperType(ResolvableType.java:452) at org.springframework.core.ResolvableType.as(ResolvableType.java:442) at org.springframework.core.ResolvableType.forClass(ResolvableType.java:1012) at org.springframework.boot.diagnostics.AbstractFailureAnalyzer.getCauseType(AbstractFailureAnalyzer.java:58) at org.springframework.boot.diagnostics.AbstractFailureAnalyzer.analyze(AbstractFailureAnalyzer.java:34) at org.springframework.boot.diagnostics.FailureAnalyzers.analyze(FailureAnalyzers.java:114) at org.springframework.boot.diagnostics.FailureAnalyzers.reportException(FailureAnalyzers.java:107) at org.springframework.boot.SpringApplication.reportFailure(SpringApplication.java:832) at org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:816) at org.springframework.boot.SpringApplication.run(SpringApplication.java:338) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1255) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1243) at com.xuecheng.manage_cms.ManageCmsApplication.main(ManageCmsApplication.java:15) Caused by: java.lang.ClassNotFoundException: org.springframework.jdbc.CannotGetJdbcConnectionException at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(CoreReflectionFactory.java:114) ... 20 common frames omitted 2019-09-27 20:40:02.282 [main] ERROR o.s.boot.SpringApplication - Application run failed java.lang.IllegalStateException: Error processing condition on org.springframework.boot.autoconfigure.data.mongo.MongoDataAutoConfiguration.mongoTemplate at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:64) at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:108) at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForBeanMethod(ConfigurationClassBeanDefinitionReader.java:179) at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForConfigurationClass(ConfigurationClassBeanDefinitionReader.java:141) at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitions(ConfigurationClassBeanDefinitionReader.java:117) at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:328) at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:233) at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:273) at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:93) at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:694) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:532) at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759) at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:395) at org.springframework.boot.SpringApplication.run(SpringApplication.java:327) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1255) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1243) at com.xuecheng.manage_cms.ManageCmsApplication.main(ManageCmsApplication.java:15) Caused by: java.lang.IllegalStateException: Failed to introspect Class [org.springframework.boot.autoconfigure.data.mongo.MongoDataAutoConfiguration] from ClassLoader [sun.misc.Launcher$AppClassLoader@18b4aac2] at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:659) at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:556) at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:541) at org.springframework.util.ReflectionUtils.getUniqueDeclaredMethods(ReflectionUtils.java:599) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getTypeForFactoryMethod(AbstractAutowireCapableBeanFactory.java:724) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.determineTargetType(AbstractAutowireCapableBeanFactory.java:665) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.predictBeanType(AbstractAutowireCapableBeanFactory.java:633) at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:1489) at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:1012) at org.springframework.boot.autoconfigure.condition.BeanTypeRegistry.addBeanTypeForNonAliasDefinition(BeanTypeRegistry.java:164) at org.springframework.boot.autoconfigure.condition.BeanTypeRegistry.addBeanType(BeanTypeRegistry.java:153) at org.springframework.boot.autoconfigure.condition.BeanTypeRegistry.updateTypesIfNecessary(BeanTypeRegistry.java:203) at org.springframework.boot.autoconfigure.condition.BeanTypeRegistry.getNamesForType(BeanTypeRegistry.java:115) at org.springframework.boot.autoconfigure.condition.OnBeanCondition.collectBeanNamesForType(OnBeanCondition.java:265) at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getBeanNamesForType(OnBeanCondition.java:254) at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getMatchingBeans(OnBeanCondition.java:196) at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getMatchOutcome(OnBeanCondition.java:116) at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:47) ... 17 common frames omitted Caused by: java.lang.NoClassDefFoundError: org/springframework/data/mongodb/core/convert/MongoCustomConversions at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.getDeclaredMethods(Class.java:1975) at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:641) ... 34 common frames omitted Caused by: java.lang.ClassNotFoundException: org.springframework.data.mongodb.core.convert.MongoCustomConversions at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 38 common frames omitted 2019-09-27 20:40:02.287 [main] INFO o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext - Closing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@59662a0b: startup date [Fri Sep 27 20:39:59 CST 2019]; root of context hierarchy 2019-09-27 20:40:02.293 [main] WARN o.s.boot.SpringApplication - Unable to close ApplicationContext java.lang.IllegalStateException: Failed to introspect Class [org.springframework.boot.autoconfigure.data.mongo.MongoDataAutoConfiguration] from ClassLoader [sun.misc.Launcher$AppClassLoader@18b4aac2] at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:659) at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:556) at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:541) at org.springframework.util.ReflectionUtils.getUniqueDeclaredMethods(ReflectionUtils.java:599) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getTypeForFactoryMethod(AbstractAutowireCapableBeanFactory.java:724) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.determineTargetType(AbstractAutowireCapableBeanFactory.java:665) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.predictBeanType(AbstractAutowireCapableBeanFactory.java:633) at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:1489) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doGetBeanNamesForType(DefaultListableBeanFactory.java:420) at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:390) at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeansOfType(DefaultListableBeanFactory.java:511) at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeansOfType(DefaultListableBeanFactory.java:503) at org.springframework.context.support.AbstractApplicationContext.getBeansOfType(AbstractApplicationContext.java:1198) at org.springframework.boot.SpringApplication.getExitCodeFromMappedException(SpringApplication.java:889) at org.springframework.boot.SpringApplication.getExitCodeFromException(SpringApplication.java:875) at org.springframework.boot.SpringApplication.handleExitCode(SpringApplication.java:861) at org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:810) at org.springframework.boot.SpringApplication.run(SpringApplication.java:338) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1255) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1243) at com.xuecheng.manage_cms.ManageCmsApplication.main(ManageCmsApplication.java:15) Caused by: java.lang.NoClassDefFoundError: org/springframework/data/mongodb/core/convert/MongoCustomConversions at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.getDeclaredMethods(Class.java:1975) at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:641) ... 20 common frames omitted Caused by: java.lang.ClassNotFoundException: org.springframework.data.mongodb.core.convert.MongoCustomConversions at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 24 common frames omitted Process finished with exit code 1 ``` **application.yml配置文件** **server: port: 31001 spring: application: name: service-manager-test data: mongodb: uri: mongodb://localhost:27017 database: test authentication-database: admin username: root password: root**
springboot 项目使用mongotemplate 查询mongo数据库,查询字段和Java实体bean不匹配。
springboot 项目使用mongotemplate 查询mongo数据库,查询字段和Java实体bean不匹配。 项目mongo数据库中使用的字段是首字母大写的。 ``` 其中某一条记录 { "_id" : ObjectId("5cdb896d20157b3976996d9b"), "Date" : "2019-07-77", "IP" : "11.11.111.111", "SessionID" : "136****888", "StartTime" : "14:37:06" } ``` 定义的实体类: ``` @Data @Document(collection = "AFVRT") public class AFVRT { @Id private String id; @Field("Date") private String Date; @Field("IP") private String IP; @Field("SessionID") private String SessionID; @Field("StartTime") private String StartTime; @Field("EndTime") private String EndTime; } ``` 查询方法: ``` public class FaceSystemAfvController { @Autowired private MongoTemplate mongoTemplate; @Autowired private StringRedisTemplate stringRedisTemplate; @RequestMapping("/sayHello") public String sayHello() throws ParseException { Query query = new Query(); query = new Query(Criteria.where("SessionID").is("AFV2464F31B000089A3")); List<AFVRT> list = mongoServerTemplate.find(query, AFVRT.class); return "Hello World"; } } ``` 报错如下: ``` 2019-05-28 17:25:17.219 ERROR 14964 --- [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.data.mapping.context.InvalidPersistentPropertyPath: No property sessionID found on com.logapi.logapifacesystem.api.afv.model.AFVRT!] with root cause org.springframework.data.mapping.context.InvalidPersistentPropertyPath: No property sessionID found on com.logapi.logapifacesystem.api.afv.model.AFVRT! at org.springframework.data.mapping.context.AbstractMappingContext.createPersistentPropertyPath(AbstractMappingContext.java:318) ~[spring-data-commons-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mapping.context.AbstractMappingContext.lambda$getPersistentPropertyPath$1(AbstractMappingContext.java:287) ~[spring-data-commons-2.0.9.RELEASE.jar:2.0.9.RELEASE] at java.util.concurrent.ConcurrentMap.computeIfAbsent(ConcurrentMap.java:324) ~[na:1.8.0_191] at org.springframework.data.mapping.context.AbstractMappingContext.getPersistentPropertyPath(AbstractMappingContext.java:286) ~[spring-data-commons-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mapping.context.AbstractMappingContext.getPersistentPropertyPath(AbstractMappingContext.java:259) ~[spring-data-commons-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mongodb.core.convert.QueryMapper$MetadataBackedField.getPath(QueryMapper.java:937) ~[spring-data-mongodb-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mongodb.core.convert.QueryMapper$MetadataBackedField.<init>(QueryMapper.java:820) ~[spring-data-mongodb-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mongodb.core.convert.QueryMapper$MetadataBackedField.<init>(QueryMapper.java:797) ~[spring-data-mongodb-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mongodb.core.convert.QueryMapper.createPropertyField(QueryMapper.java:245) ~[spring-data-mongodb-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mongodb.core.convert.QueryMapper.getMappedObject(QueryMapper.java:139) ~[spring-data-mongodb-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2237) ~[spring-data-mongodb-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2227) ~[spring-data-mongodb-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mongodb.core.MongoTemplate.find(MongoTemplate.java:770) ~[spring-data-mongodb-2.0.9.RELEASE.jar:2.0.9.RELEASE] at org.springframework.data.mongodb.core.MongoTemplate.find(MongoTemplate.java:756) ~[spring-data-mongodb-2.0.9.RELEASE.jar:2.0.9.RELEASE] at com.logapi.logapifacesystem.api.afv.controller.FaceSystemAfvController.sayHello(FaceSystemAfvController.java:69) ~[classes/:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_191] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209) ~[spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) ~[spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) ~[spring-webmvc-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877) ~[spring-webmvc-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:783) ~[spring-webmvc-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991) ~[spring-webmvc-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925) ~[spring-webmvc-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974) ~[spring-webmvc-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:866) ~[spring-webmvc-5.0.8.RELEASE.jar:5.0.8.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851) ~[spring-webmvc-5.0.8.RELEASE.jar:5.0.8.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.5.32.jar:8.5.32] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109) ~[spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) ~[spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) ~[spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) [tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800) [tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:800) [tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1471) [tomcat-embed-core-8.5.32.jar:8.5.32] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.32.jar:8.5.32] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_191] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_191] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.32.jar:8.5.32] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_191] ``` 说明: 1,如果将实体类属性字段改成首字母小写,则不会报错。 2,去掉实体类属性的gei,set方法也不会报错 3,查询时不传入条件,直接查询所有的话也不会报错 百度找了几天也没找到答案,希望大神可以帮帮忙!!!!!!!
现在是一些关键操作要记录操作日志,但是在insert的时候,偶然会出现插入失败的问题.现在找不到问题所在?
执行代码 ``` HttpServletRequest request = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getRequest(); SsOperLog ssOperLogEntity = new SsOperLog(); ssOperLogEntity.setOperNo(Long.valueOf(IdGenarater.nextStringValue()));//操作流水号 UserInfo users = Util.getSessionUserInfo(request);//登录用户 if (users == null) {//用户还未登录 String workCode = request.getParameter("workCode"); String urlString = request.getHeader("host"); //根据域名获取部门编号 String deptCode = loginService.getDeptCodeByDomainName(urlString); users = loginService.selectByUserIdAndPassword(workCode, null, deptCode); if (users == null) { return point.proceed(); } } ssOperLogEntity.setDeptCode(users.getDeptCode());//用户机构 ssOperLogEntity.setOperId(Long.valueOf(users.getOperId()));//用户id ssOperLogEntity.setOperName(users.getOperName());//操作员 ssOperLogEntity.setDeptName(users.getDeptName()); ssOperLogEntity.setOperDate(new Date(System.currentTimeMillis()));//操作日期 ssOperLogEntity.setOperTime(new Date(System.currentTimeMillis()));//操作时间 String rquestName = getMthodRemark(point);//操作名称(对应注解里面的remark信息) String operType = getMthodOperType(point);//操作类型(对应注解里面的opertype信息) String processMethod = point.getSignature().getName();//请求处理方法名 ssOperLogEntity.setReruestId(request.getRequestURI());//请求url ssOperLogEntity.setRquestName(rquestName); if(operType.length()>1){ //获取大类类型 ssOperLogEntity.setCategories(operType.substring(0,1)); } ssOperLogEntity.setProcessMethod(processMethod); ssOperLogEntity.setOperType(operType); ssOperLogEntity.setLocalAddress(request.getRemoteAddr()); if (!"/aa/login/checkLogin.do".equals(request.getRequestURI())) { String requestParams = JSONObject.toJSONString(request.getParameterMap()); ssOperLogEntity.setRequestParams(requestParams); }//请求参数 Object[] method_param = null; Object object; try { method_param = point.getArgs(); //获取方法参数 object = point.proceed(); } catch (Exception e) { ssOperLogEntity.setOperState(OperateStatusEnum.FAIL.getCode());//操作失败 ssOperLogEntity.setExceptionMesg(e.getMessage()); if (e.getMessage() == null) { ssOperLogEntity.setExceptionMesg(e.toString()); } mongoTemplate.insert(ssOperLogEntity); // ssOperLogMapper.insert(ssOperLogEntity); throw e; } ssOperLogEntity.setOperState(OperateStatusEnum.SUCC.getCode());//操作成功 mongoTemplate.insert(ssOperLogEntity); // 这里报错 // ssOperLogMapper.insert(ssOperLogEntity); return object; } ``` 异常信息:appserver-b4 等同于ip地址 ``` org.springframework.dao.DataAccessResourceFailureException: Operation on server appserver-b4:27017 failed; nested exception is com.mongodb.MongoException$Network: Operation on server appserver-b4:27017 failed at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:77) ~[spring-data-mongodb-1.10.15.RELEASE.jar:na] at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2128) ~[spring-data-mongodb-1.10.15.RELEASE.jar:na] at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:461) ~[spring-data-mongodb-1.10.15.RELEASE.jar:na] at org.springframework.data.mongodb.core.MongoTemplate.insertDBObject(MongoTemplate.java:1026) ~[spring-data-mongodb-1.10.15.RELEASE.jar:na] at org.springframework.data.mongodb.core.MongoTemplate.doInsert(MongoTemplate.java:835) ~[spring-data-mongodb-1.10.15.RELEASE.jar:na] at org.springframework.data.mongodb.core.MongoTemplate.insert(MongoTemplate.java:776) ~[spring-data-mongodb-1.10.15.RELEASE.jar:na] at org.springframework.data.mongodb.core.MongoTemplate.insert(MongoTemplate.java:767) ~[spring-data-mongodb-1.10.15.RELEASE.jar:na] at com.wx.app.ygp.service.log.LogService.doBefore(LogService.java:115) ~[classes/:na] at sun.reflect.GeneratedMethodAccessor867.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_162] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_162] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) ~[spring-aop-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) ~[spring-aop-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) ~[spring-aop-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168) ~[spring-aop-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:654) ~[spring-aop-4.2.3.RELEASE.jar:4.2.3.RELEASE] at com.wx.app.ygp.action.system.LoginController$$EnhancerBySpringCGLIB$$3d4c3a49.checkLogin(<generated>) ~[classes/:na] at sun.reflect.GeneratedMethodAccessor1317.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_162] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_162] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:222) ~[spring-web-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137) ~[spring-web-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) ~[spring-webmvc-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:814) ~[spring-webmvc-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:737) ~[spring-webmvc-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) [spring-webmvc-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) [spring-webmvc-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) [spring-webmvc-4.2.3.RELEASE.jar:4.2.3.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) [servlet-api.jar:na] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.2.3.RELEASE.jar:4.2.3.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) [servlet-api.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) [catalina.jar:8.0.44] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [catalina.jar:8.0.44] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-websocket.jar:8.0.44] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [catalina.jar:8.0.44] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [catalina.jar:8.0.44] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121) [spring-web-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [catalina.jar:8.0.44] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [catalina.jar:8.0.44] at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:167) [spring-session-1.3.1.RELEASE.jar:na] at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:80) [spring-session-1.3.1.RELEASE.jar:na] at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) [spring-web-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) [spring-web-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [catalina.jar:8.0.44] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [catalina.jar:8.0.44] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) [catalina.jar:8.0.44] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94) [catalina.jar:8.0.44] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) [catalina.jar:8.0.44] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) [catalina.jar:8.0.44] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [catalina.jar:8.0.44] at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) [catalina.jar:8.0.44] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [catalina.jar:8.0.44] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) [catalina.jar:8.0.44] at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132) [tomcat-coyote.jar:8.0.44] at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) [tomcat-coyote.jar:8.0.44] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1533) [tomcat-coyote.jar:8.0.44] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1489) [tomcat-coyote.jar:8.0.44] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_162] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_162] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:8.0.44] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162] Caused by: com.mongodb.MongoException$Network: Operation on server appserver-b4:27017 failed at com.mongodb.DBTCPConnector.doOperation(DBTCPConnector.java:215) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollectionImpl.writeWithCommandProtocol(DBCollectionImpl.java:567) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollectionImpl.insertWithCommandProtocol(DBCollectionImpl.java:528) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollectionImpl.insertImpl(DBCollectionImpl.java:206) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollection.insert(DBCollection.java:176) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollection.insert(DBCollection.java:93) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollection.insert(DBCollection.java:78) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollection.insert(DBCollection.java:120) ~[mongo-java-driver-2.14.3.jar:na] at org.springframework.data.mongodb.core.MongoTemplate$9.doInCollection(MongoTemplate.java:1031) ~[spring-data-mongodb-1.10.15.RELEASE.jar:na] at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:459) ~[spring-data-mongodb-1.10.15.RELEASE.jar:na] ... 63 common frames omitted Caused by: java.net.SocketException: Broken pipe (Write failed) at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.8.0_162] at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111) ~[na:1.8.0_162] at java.net.SocketOutputStream.write(SocketOutputStream.java:155) ~[na:1.8.0_162] at org.bson.io.PoolOutputBuffer.pipe(PoolOutputBuffer.java:153) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollectionImpl.sendWriteCommandMessage(DBCollectionImpl.java:639) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollectionImpl.access$300(DBCollectionImpl.java:50) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollectionImpl$4.execute(DBCollectionImpl.java:576) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBCollectionImpl$4.execute(DBCollectionImpl.java:567) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBPort.doOperation(DBPort.java:187) ~[mongo-java-driver-2.14.3.jar:na] at com.mongodb.DBTCPConnector.doOperation(DBTCPConnector.java:208) ~[mongo-java-driver-2.14.3.jar:na] ... 72 common frames omitted ``` 配置MongoDB文件: ``` mongo.hostport=127.0.0.1:27017 mongo.connectionsPerHost=100 mongo.threadsAllowedToBlockForConnectionMultiplier=6 mongo.dataName=ygp mongo.username=ygp mongo.password=ygp_admin mongo.connectTimeout=60000 mongo.maxWaitTime=120000 mongo.autoConnectRetry=true mongo.socketKeepAlive=true mongo.socketTimeout=1500 mongo.slaveOk=true <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:mongo="http://www.springframework.org/schema/data/mongo" xsi:schemaLocation="http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd http://www.springframework.org/schema/data/mongo http://www.springframework.org/schema/data/mongo/spring-mongo-1.0.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd"> <!--设置用户验证--> <bean id="userCredentials" class="org.springframework.data.authentication.UserCredentials"> <constructor-arg name="username" value="${mongo.username}" /> <constructor-arg name="password" value="${mongo.password}" /> </bean> <!-- 定义mongo对象,对应的是mongodb官方jar包中的Mongo,replica-set设置集群副本的ip地址和端口 --> <mongo:mongo id="mongo" replica-set="${mongo.hostport}"> <!-- 一些连接属性的设置 --> <mongo:options connections-per-host="${mongo.connectionsPerHost}" threads-allowed-to-block-for-connection-multiplier="${mongo.threadsAllowedToBlockForConnectionMultiplier}" connect-timeout="${mongo.connectTimeout}" max-wait-time="${mongo.maxWaitTime}" auto-connect-retry="${mongo.autoConnectRetry}" socket-keep-alive="${mongo.socketKeepAlive}" socket-timeout="${mongo.socketTimeout}" slave-ok="${mongo.slaveOk}" write-number="1" write-timeout="0" write-fsync="true" /> </mongo:mongo> <!-- mongo的工厂,通过它来取得mongo实例,dbname为mongodb的数据库名,没有的话会自动创建 --> <bean id="mongoDbFactory" class="org.springframework.data.mongodb.core.SimpleMongoDbFactory"> <constructor-arg ref="mongo" /> <constructor-arg value="${mongo.dataName}" /> <constructor-arg ref="userCredentials" /> </bean> <bean id="mappingContext" class="org.springframework.data.mongodb.core.mapping.MongoMappingContext" /> <!--排除 _class --> <bean id="defaultMongoTypeMapper" class="org.springframework.data.mongodb.core.convert.DefaultMongoTypeMapper"> <constructor-arg name="typeKey"> <null /> </constructor-arg> </bean> <!-- collection的映射 --> <bean id="mappingMongoConverter" class="org.springframework.data.mongodb.core.convert.MappingMongoConverter"> <constructor-arg name="mongoDbFactory" ref="mongoDbFactory" /> <constructor-arg name="mappingContext" ref="mappingContext" /> <property name="typeMapper" ref="defaultMongoTypeMapper" /> </bean> <!-- mongodb的主要操作对象,所有对mongodb的增删改查的操作都是通过它完成 --> <bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"> <constructor-arg name="mongoDbFactory" ref="mongoDbFactory" /> <constructor-arg name="mongoConverter" ref="mappingMongoConverter" /> </bean> </beans> ```
flask蓝图模式下,怎么使用pymongo
``` from flask import Flask from flask-pymongo import PyMongo app = Flask(__name__) app.config.update( MONGO_HOST='localhost', MONGO_PORT=27017, MONGO_USERNAME='bjhee', MONGO_PASSWORD='111111', MONGO_DBNAME='flask' ) mongo = PyMongo(app) ``` 这是直接在入口文件里写的方法 现在加入里蓝图 home.py ``` from flask import Blueprint import mysql.connector, logging from flask_pymongo import PyMongo import app home = Blueprint('home', __name__, url_prefix='/home') mongo = PyMongo(app) // 这里应该怎么写 因为不存在app @home.route('/test') def move_tickets_to_mongo(): res = mongo.db.acct_data_logs.find({}) ``` 这里会报错, 因为找不到app
mongo连接数和maxCons设置为20000,还是频繁报连接数不够异常
异常和连接池的配置如下: ``` com.mongodb.DBPortPool$SemaphoresOut: Out of semaphores to get db connection at com.mongodb.DBPortPool.get(DBPortPool.java:206) at com.mongodb.DBTCPConnector$MyPort.get(DBTCPConnector.java:437) at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:277) at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:256) at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:289) at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:274) at com.mongodb.DBCollection.findOne(DBCollection.java:710) at com.mongodb.DBCollection.findOne(DBCollection.java:652) at com.hzwaso.miniapp.business.action.WeChatGraphicEditingPushAction.getTitle(WeChatGraphicEditingPushAction.java:1291) at com.hzwaso.miniapp.business.action.WeChatGraphicEditingPushAction.queryDetail(WeChatGraphicEditingPushAction.java:135) at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at ognl.OgnlRuntime.invokeMethod(OgnlRuntime.java:892) at ognl.OgnlRuntime.callAppropriateMethod(OgnlRuntime.java:1294) at ognl.ObjectMethodAccessor.callMethod(ObjectMethodAccessor.java:68) at com.opensymphony.xwork2.ognl.accessor.XWorkMethodAccessor.callMethodWithDebugInfo(XWorkMethodAccessor.java:117) at com.opensymphony.xwork2.ognl.accessor.XWorkMethodAccessor.callMethod(XWorkMethodAccessor.java:108) at ognl.OgnlRuntime.callMethod(OgnlRuntime.java:1370) at ognl.ASTMethod.getValueBody(ASTMethod.java:91) at ognl.SimpleNode.evaluateGetValueBody(SimpleNode.java:212) at ognl.SimpleNode.getValue(SimpleNode.java:258) at ognl.Ognl.getValue(Ognl.java:467) at ognl.Ognl.getValue(Ognl.java:431) at com.opensymphony.xwork2.ognl.OgnlUtil$3.execute(OgnlUtil.java:352) at com.opensymphony.xwork2.ognl.OgnlUtil.compileAndExecuteMethod(OgnlUtil.java:404) at com.opensymphony.xwork2.ognl.OgnlUtil.callMethod(OgnlUtil.java:350) at com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:430) at com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:290) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:251) at org.apache.struts2.interceptor.DeprecationInterceptor.intercept(DeprecationInterceptor.java:41) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at org.apache.struts2.interceptor.debugging.DebuggingInterceptor.intercept(DebuggingInterceptor.java:256) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.DefaultWorkflowInterceptor.doIntercept(DefaultWorkflowInterceptor.java:168) at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:98) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.validator.ValidationInterceptor.doIntercept(ValidationInterceptor.java:265) at org.apache.struts2.interceptor.validation.AnnotationValidationInterceptor.doIntercept(AnnotationValidationInterceptor.java:76) at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:98) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.ConversionErrorInterceptor.intercept(ConversionErrorInterceptor.java:138) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:229) at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:98) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:229) at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:98) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.StaticParametersInterceptor.intercept(StaticParametersInterceptor.java:191) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at org.apache.struts2.interceptor.MultiselectInterceptor.intercept(MultiselectInterceptor.java:73) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at org.apache.struts2.interceptor.DateTextFieldInterceptor.intercept(DateTextFieldInterceptor.java:125) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at org.apache.struts2.interceptor.CheckboxInterceptor.intercept(CheckboxInterceptor.java:91) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at org.apache.struts2.interceptor.FileUploadInterceptor.intercept(FileUploadInterceptor.java:253) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.ModelDrivenInterceptor.intercept(ModelDrivenInterceptor.java:100) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.ScopedModelDrivenInterceptor.intercept(ScopedModelDrivenInterceptor.java:141) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.ChainingInterceptor.intercept(ChainingInterceptor.java:145) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.PrepareInterceptor.doIntercept(PrepareInterceptor.java:171) at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:98) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.I18nInterceptor.intercept(I18nInterceptor.java:140) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at org.apache.struts2.interceptor.ServletConfigInterceptor.intercept(ServletConfigInterceptor.java:164) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.AliasInterceptor.intercept(AliasInterceptor.java:193) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at com.opensymphony.xwork2.interceptor.ExceptionMappingInterceptor.intercept(ExceptionMappingInterceptor.java:189) at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:245) at org.apache.struts2.impl.StrutsActionProxy.execute(StrutsActionProxy.java:54) at org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:575) at org.apache.struts2.dispatcher.ng.ExecuteOperations.executeAction(ExecuteOperations.java:81) at org.apache.struts2.dispatcher.ng.filter.StrutsPrepareAndExecuteFilter.doFilter(StrutsPrepareAndExecuteFilter.java:99) at com.hzwaso.miniapp.business.filter.EditorStrutsFilter.doFilter(EditorStrutsFilter.java:21) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.hzwaso.miniapp.common.filter.EncodedFilter.doFilter(EncodedFilter.java:24) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.hzwaso.miniapp.business.filter.WeChatFilter.doFilter(WeChatFilter.java:844) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) public class MongoUtil { private static Mongo mongo = null; private static DB db = null; private final static int POOLSIZE = 5000;// 连接数量 private final static int BLOCKSIZE = 1000; // 等待队列长度 private static Log log = LogFactory.getLog(MongoUtil.class); /** * 根据名称获取DB,相当于是连接 * * @param dbName * @return */ public static DB getMongoDB() { if (mongo == null) { init(); db = mongo.getDB(ConfigUtils.systemConf("mongo.db")); db.authenticate(ConfigUtils.systemConf("mongo.user"), ConfigUtils.systemConf("mongo.pwd").toCharArray()); } return db; } private static void init() { String host = ConfigUtils.systemConf("mongo.ip"); int port = new Integer("27017"); try { mongo = new Mongo(host, port); MongoOptions opt = mongo.getMongoOptions(); opt.connectionsPerHost = POOLSIZE; opt.threadsAllowedToBlockForConnectionMultiplier = BLOCKSIZE; } catch (UnknownHostException e) { log.error(e); } catch (MongoException e) { mongo.close(); mongo = null; log.error(e); } } } ```
如何将客户端(ubuntu18.04 nxlog+graylog-collector-sidecar)日志传给服务器端(centOS7 graylog2.0+mongo+elasticesearch)
已经完成的工作 1、服务器端graylog的web已经可以访问了 2、客户端graylog-collector-sidecar也安装配置完了,启动正常 3、修改了配置文件/etc/graylog/collector-sidecar/collector_sidecar.yml 如下: ``` server_url: http://192.168.1.7:12900 #graylog端的IP node_id: graylog-collector-sidecar collector_id: file:/etc/graylog/collector-sidecar/collector-id tags: - linux - apache - redis update_interval: 10 log_path: /var/log/graylog/collector-sidecar backends: - name: nxlog enabled: true binary_path: /usr/bin/nxlog configuration_path: /etc/graylog/collector-sidecar/generated/nxlog.conf ``` 4、nxlog安装好了,修改配置文件/etc/nxlog/nxlog.conf 如下,启动失败 ``` ######################################## # Modules # ######################################## #<Extension _syslog> # Module xm_syslog #</Extension> <Input in1> Module im_file File "/var/tmp/opencanary.log" #实际想要收集的log文件 SavePos TRUE </Input> #<Input in2> # Module im_tcp # Port 514 #</Input> <Output fileout1> Module om_udp Host 192.168.1.7 Port 12201 #与graylog的input端口对应 </Output> #<Output fileout2> # Module om_file # File "/var/log/nxlog/logmsg2.txt" #</Output> ######################################## # Routes # ######################################## <Route 1> Path in1 => fileout1 </Route> #<Route tcproute> # Path in2 => fileout2 #</Route> ``` 5、运行systemctl start nxlog.service报错, ![图片说明](https://img-ask.csdn.net/upload/201912/24/1577199315_599493.png) 问题: 怎么才能让graylog收集到日志,请教各位大佬。
mongo morphia 日期分组
使用java代码 mongo 的 morphia 日期分组统计:如:2017-03-23 现在是:2017-03-23 14:23:23 是怎么写的?求指教 Datastore datastore = simpleMongoDBDao.getDB(); Query<BcsBillListDOC> query = datastore.createQuery(BcsBillListDOC.class); query=query.filter("createTime >= ", DateUtil.getDateBystr(DateUtil.getSpecifiedDayBefore(date,7) + " 00:00:00")); query=query.filter("payResult = ", 1 ); AggregationPipeline pipeline = datastore.createAggregation(BcsBillListDOC.class) .match(query) .group( id( # **这里不知道怎么写** ), grouping("tradeMoney",sum("tradeMoney")), grouping("createTime",first("createTime")) ).sort(Sort.descending("createTime")); Iterator<BillListGroupDOC> iterator = pipeline.aggregate(BillListGroupDOC.class);
mongo已经设置最大连接了,还是频繁报连接数不够
这个是获取连接的代码 ``` public class MongoUtil { private static Mongo mongo = null; private static DB db = null; private final static int POOLSIZE = 5000;// 连接数量 private final static int BLOCKSIZE = 1000; // 等待队列长度 private static Log log = LogFactory.getLog(MongoUtil.class); /** * 根据名称获取DB,相当于是连接 * * @param dbName * @return */ public static DB getMongoDB() { if (mongo == null) { // 初始化 init(); db = mongo.getDB(ConfigUtils.systemConf("mongo.db")); db.authenticate(ConfigUtils.systemConf("mongo.user"), ConfigUtils.systemConf("mongo.pwd").toCharArray()); } return db; } private static void init() { String host = ConfigUtils.systemConf("mongo.ip");// 主机名 int port = new Integer("27017");// 端口 // 其他参数根据实际情况进行添加 try { mongo = new Mongo(host, port); MongoOptions opt = mongo.getMongoOptions(); opt.connectionsPerHost = POOLSIZE; opt.threadsAllowedToBlockForConnectionMultiplier = BLOCKSIZE; } catch (UnknownHostException e) { log.error(e); } catch (MongoException e) { log.error(e); } } } ``` 频繁报:Out of semaphores to get db connection
Mongo怎么限制每个分组下返回的文档数?
我现在有一个contact集合,代表所有联系人。现在按照每个联系人的首字母分组,每个首字母下的联系人限制在5个以内。举个例子吧: 这个一个联系人表: Kobe Kevin Kolon Kaidi Kaisin Kama Branyt Bolon Boo 我要得到的效果是这样的,以A开头的联系人有六个我只能返回5个,以B开头的联系人有3个全部会被返回。返回结果如下 K: Kobe Kevin Kolon Kaidi Kaisin B: Branyt Bolon Boo 采用mongo的aggregate的写法,或者其他写法也行
MongoDB 关于mongo.exe黑色框程序闪退
如果又跟我一样的,网上的教程几乎都试了个遍还没有用的。安装低一版本的MongoDB试一下,之前我是在官网下载4.2版本的,怎么样都没有。后来我下了4.0.12版本的(要安装在系统默认路径呦)就好了。太难了 试了好久 我先是在MongoDB bin目录下双击mongo.exe文件结果出现黑框闪退 接着我又使用cmd来运行mongo ``` C:\Program Files\MongoDB\bin>mongo ``` 结果出现了一下结果 2019-09-14T18:56:54.663+0800 F CONTROL [main] *** unhandled exception 0xE06D7363 at 0x00007FFFF0CAA839, terminating ``` connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb Implicit session: session { "id" : UUID("43123533-717d-4ec1-9e4f-fb95154c2ceb") } MongoDB server version: 4.2.0 Server has startup warnings: 2019-09-14T16:46:16.226+0800 I CONTROL [initandlisten] 2019-09-14T16:46:16.226+0800 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database. 2019-09-14T16:46:16.226+0800 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted. 2019-09-14T16:46:16.226+0800 I CONTROL [initandlisten] --- Enable MongoDB's free cloud-based monitoring service, which will then receive and display metrics about your deployment (disk utilization, CPU, operation statistics, etc). The monitoring data will be available on a MongoDB website with a unique URL accessible to you and anyone you share the URL with. MongoDB may use this information to make product improvements and to suggest MongoDB products and deployment options to you. To enable free monitoring, run the following command: db.enableFreeMonitoring() To permanently disable this reminder, run the following command: db.disableFreeMonitoring() --- 2019-09-14T18:56:54.663+0800 F CONTROL [main] *** unhandled exception 0xE06D7363 at 0x00007FFFF0CAA839, terminating 2019-09-14T18:56:54.665+0800 F CONTROL [main] *** stack trace for unhandled exception: 2019-09-14T18:56:54.667+0800 I - [main] KERNELBASE.dll RaiseException+0x69 VCRUNTIME140.dll CxxThrowException+0xad mongo.exe MallocExtension::GetEstimatedAllocatedSize+0x175210 mongo.exe MallocExtension::GetEstimatedAllocatedSize+0x174fef mongo.exe tc_mallopt+0xa52c mongo.exe mozilla::detail::MutexImpl::~MutexImpl+0x866f mongo.exe mozilla::detail::MutexImpl::~MutexImpl+0x2317 mongo.exe mozilla::detail::MutexImpl::~MutexImpl+0x8de5 mongo.exe tcmalloc::Log+0xaaf04 KERNEL32.DLL BaseThreadInitThunk+0x14 2019-09-14T18:56:54.668+0800 I CONTROL [main] writing minidump diagnostic file C:\Program Files\MongoDB\bin\mongo.2019-09-14T10-56-54.mdmp 2019-09-14T18:56:54.851+0800 F CONTROL [main] *** immediate exit due to unhandled exception C:\Program Files\MongoDB\bin> ``` ![图片说明](https://img-ask.csdn.net/upload/201909/14/1568459706_295567.png)
爬虫福利二 之 妹子图网MM批量下载
爬虫福利一:27报网MM批量下载 点击 看了本文,相信大家对爬虫一定会产生强烈的兴趣,激励自己去学习爬虫,在这里提前祝:大家学有所成! 目标网站:妹子图网 环境:Python3.x 相关第三方模块:requests、beautifulsoup4 Re:各位在测试时只需要将代码里的变量path 指定为你当前系统要保存的路径,使用 python xxx.py 或IDE运行即可。 ...
Java学习的正确打开方式
在博主认为,对于入门级学习java的最佳学习方法莫过于视频+博客+书籍+总结,前三者博主将淋漓尽致地挥毫于这篇博客文章中,至于总结在于个人,实际上越到后面你会发现学习的最好方式就是阅读参考官方文档其次就是国内的书籍,博客次之,这又是一个层次了,这里暂时不提后面再谈。博主将为各位入门java保驾护航,各位只管冲鸭!!!上天是公平的,只要不辜负时间,时间自然不会辜负你。 何谓学习?博主所理解的学习,它是一个过程,是一个不断累积、不断沉淀、不断总结、善于传达自己的个人见解以及乐于分享的过程。
大学四年自学走来,这些私藏的实用工具/学习网站我贡献出来了
大学四年,看课本是不可能一直看课本的了,对于学习,特别是自学,善于搜索网上的一些资源来辅助,还是非常有必要的,下面我就把这几年私藏的各种资源,网站贡献出来给你们。主要有:电子书搜索、实用工具、在线视频学习网站、非视频学习网站、软件下载、面试/求职必备网站。 注意:文中提到的所有资源,文末我都给你整理好了,你们只管拿去,如果觉得不错,转发、分享就是最大的支持了。 一、电子书搜索 对于大部分程序员...
linux系列之常用运维命令整理笔录
本博客记录工作中需要的linux运维命令,大学时候开始接触linux,会一些基本操作,可是都没有整理起来,加上是做开发,不做运维,有些命令忘记了,所以现在整理成博客,当然vi,文件操作等就不介绍了,慢慢积累一些其它拓展的命令,博客不定时更新 free -m 其中:m表示兆,也可以用g,注意都要小写 Men:表示物理内存统计 total:表示物理内存总数(total=used+free) use...
比特币原理详解
一、什么是比特币 比特币是一种电子货币,是一种基于密码学的货币,在2008年11月1日由中本聪发表比特币白皮书,文中提出了一种去中心化的电子记账系统,我们平时的电子现金是银行来记账,因为银行的背后是国家信用。去中心化电子记账系统是参与者共同记账。比特币可以防止主权危机、信用风险。其好处不多做赘述,这一层面介绍的文章很多,本文主要从更深层的技术原理角度进行介绍。 二、问题引入 假设现有4个人...
程序员接私活怎样防止做完了不给钱?
首先跟大家说明一点,我们做 IT 类的外包开发,是非标品开发,所以很有可能在开发过程中会有这样那样的需求修改,而这种需求修改很容易造成扯皮,进而影响到费用支付,甚至出现做完了项目收不到钱的情况。 那么,怎么保证自己的薪酬安全呢? 我们在开工前,一定要做好一些证据方面的准备(也就是“讨薪”的理论依据),这其中最重要的就是需求文档和验收标准。一定要让需求方提供这两个文档资料作为开发的基础。之后开发...
网页实现一个简单的音乐播放器(大佬别看。(⊙﹏⊙))
今天闲着无事,就想写点东西。然后听了下歌,就打算写个播放器。 于是乎用h5 audio的加上js简单的播放器完工了。 演示地点演示 html代码如下` music 这个年纪 七月的风 音乐 ` 然后就是css`*{ margin: 0; padding: 0; text-decoration: none; list-...
Python十大装B语法
Python 是一种代表简单思想的语言,其语法相对简单,很容易上手。不过,如果就此小视 Python 语法的精妙和深邃,那就大错特错了。本文精心筛选了最能展现 Python 语法之精妙的十个知识点,并附上详细的实例代码。如能在实战中融会贯通、灵活使用,必将使代码更为精炼、高效,同时也会极大提升代码B格,使之看上去更老练,读起来更优雅。
数据库优化 - SQL优化
以实际SQL入手,带你一步一步走上SQL优化之路!
2019年11月中国大陆编程语言排行榜
2019年11月2日,我统计了某招聘网站,获得有效程序员招聘数据9万条。针对招聘信息,提取编程语言关键字,并统计如下: 编程语言比例 rank pl_ percentage 1 java 33.62% 2 cpp 16.42% 3 c_sharp 12.82% 4 javascript 12.31% 5 python 7.93% 6 go 7.25% 7 p...
通俗易懂地给女朋友讲:线程池的内部原理
餐盘在灯光的照耀下格外晶莹洁白,女朋友拿起红酒杯轻轻地抿了一小口,对我说:“经常听你说线程池,到底线程池到底是个什么原理?”
经典算法(5)杨辉三角
杨辉三角 是经典算法,这篇博客对它的算法思想进行了讲解,并有完整的代码实现。
腾讯算法面试题:64匹马8个跑道需要多少轮才能选出最快的四匹?
昨天,有网友私信我,说去阿里面试,彻底的被打击到了。问了为什么网上大量使用ThreadLocal的源码都会加上private static?他被难住了,因为他从来都没有考虑过这个问题。无独有偶,今天笔者又发现有网友吐槽了一道腾讯的面试题,我们一起来看看。 腾讯算法面试题:64匹马8个跑道需要多少轮才能选出最快的四匹? 在互联网职场论坛,一名程序员发帖求助到。二面腾讯,其中一个算法题:64匹...
面试官:你连RESTful都不知道我怎么敢要你?
干货,2019 RESTful最贱实践
JDK12 Collectors.teeing 你真的需要了解一下
前言 在 Java 12 里面有个非常好用但在官方 JEP 没有公布的功能,因为它只是 Collector 中的一个小改动,它的作用是 merge 两个 collector 的结果,这句话显得很抽象,老规矩,我们先来看个图(这真是一个不和谐的图????): 管道改造经常会用这个小东西,通常我们叫它「三通」,它的主要作用就是将 downstream1 和 downstre...
为啥国人偏爱Mybatis,而老外喜欢Hibernate/JPA呢?
关于SQL和ORM的争论,永远都不会终止,我也一直在思考这个问题。昨天又跟群里的小伙伴进行了一番讨论,感触还是有一些,于是就有了今天这篇文。 声明:本文不会下关于Mybatis和JPA两个持久层框架哪个更好这样的结论。只是摆事实,讲道理,所以,请各位看官勿喷。 一、事件起因 关于Mybatis和JPA孰优孰劣的问题,争论已经很多年了。一直也没有结论,毕竟每个人的喜好和习惯是大不相同的。我也看...
SQL-小白最佳入门sql查询一
不要偷偷的查询我的个人资料,即使你再喜欢我,也不要这样,真的不好;
项目中的if else太多了,该怎么重构?
介绍 最近跟着公司的大佬开发了一款IM系统,类似QQ和微信哈,就是聊天软件。我们有一部分业务逻辑是这样的 if (msgType = "文本") { // dosomething } else if(msgType = "图片") { // doshomething } else if(msgType = "视频") { // doshomething } else { // doshom...
【图解经典算法题】如何用一行代码解决约瑟夫环问题
约瑟夫环问题算是很经典的题了,估计大家都听说过,然后我就在一次笔试中遇到了,下面我就用 3 种方法来详细讲解一下这道题,最后一种方法学了之后保证让你可以让你装逼。 问题描述:编号为 1-N 的 N 个士兵围坐在一起形成一个圆圈,从编号为 1 的士兵开始依次报数(1,2,3…这样依次报),数到 m 的 士兵会被杀死出列,之后的士兵再从 1 开始报数。直到最后剩下一士兵,求这个士兵的编号。 1、方...
致 Python 初学者
欢迎来到“Python进阶”专栏!来到这里的每一位同学,应该大致上学习了很多 Python 的基础知识,正在努力成长的过程中。在此期间,一定遇到了很多的困惑,对未来的学习方向感到迷茫。我非常理解你们所面临的处境。我从2007年开始接触 python 这门编程语言,从2009年开始单一使用 python 应对所有的开发工作,直至今天。回顾自己的学习过程,也曾经遇到过无数的困难,也曾经迷茫过、困惑过。开办这个专栏,正是为了帮助像我当年一样困惑的 Python 初学者走出困境、快速成长。希望我的经验能真正帮到你
“狗屁不通文章生成器”登顶GitHub热榜,分分钟写出万字形式主义大作
一、垃圾文字生成器介绍 最近在浏览GitHub的时候,发现了这样一个骨骼清奇的雷人项目,而且热度还特别高。 项目中文名:狗屁不通文章生成器 项目英文名:BullshitGenerator 根据作者的介绍,他是偶尔需要一些中文文字用于GUI开发时测试文本渲染,因此开发了这个废话生成器。但由于生成的废话实在是太过富于哲理,所以最近已经被小伙伴们给玩坏了。 他的文风可能是这样的: 你发现,...
程序员:我终于知道post和get的区别
是一个老生常谈的话题,然而随着不断的学习,对于以前的认识有很多误区,所以还是需要不断地总结的,学而时习之,不亦说乎
GitHub标星近1万:只需5秒音源,这个网络就能实时“克隆”你的声音
作者 | Google团队 译者 | 凯隐 编辑 | Jane 出品 | AI科技大本营(ID:rgznai100) 本文中,Google 团队提出了一种文本语音合成(text to speech)神经系统,能通过少量样本学习到多个不同说话者(speaker)的语音特征,并合成他们的讲话音频。此外,对于训练时网络没有接触过的说话者,也能在不重新训练的情况下,仅通过未知...
《程序人生》系列-这个程序员只用了20行代码就拿了冠军
你知道的越多,你不知道的越多 点赞再看,养成习惯GitHub上已经开源https://github.com/JavaFamily,有一线大厂面试点脑图,欢迎Star和完善 前言 这一期不算《吊打面试官》系列的,所有没前言我直接开始。 絮叨 本来应该是没有这期的,看过我上期的小伙伴应该是知道的嘛,双十一比较忙嘛,要值班又要去帮忙拍摄年会的视频素材,还得搞个程序员一天的Vlog,还要写BU...
加快推动区块链技术和产业创新发展,2019可信区块链峰会在京召开
11月8日,由中国信息通信研究院、中国通信标准化协会、中国互联网协会、可信区块链推进计划联合主办,科技行者协办的2019可信区块链峰会将在北京悠唐皇冠假日酒店开幕。   区块链技术被认为是继蒸汽机、电力、互联网之后,下一代颠覆性的核心技术。如果说蒸汽机释放了人类的生产力,电力解决了人类基本的生活需求,互联网彻底改变了信息传递的方式,区块链作为构造信任的技术有重要的价值。   1...
程序员把地府后台管理系统做出来了,还有3.0版本!12月7号最新消息:已在开发中有github地址
第一幕:缘起 听说阎王爷要做个生死簿后台管理系统,我们派去了一个程序员…… 996程序员做的梦: 第一场:团队招募 为了应对地府管理危机,阎王打算找“人”开发一套地府后台管理系统,于是就在地府总经办群中发了项目需求。 话说还是中国电信的信号好,地府都是满格,哈哈!!! 经常会有外行朋友问:看某网站做的不错,功能也简单,你帮忙做一下? 而这次,面对这样的需求,这个程序员...
网易云6亿用户音乐推荐算法
网易云音乐是音乐爱好者的集聚地,云音乐推荐系统致力于通过 AI 算法的落地,实现用户千人千面的个性化推荐,为用户带来不一样的听歌体验。 本次分享重点介绍 AI 算法在音乐推荐中的应用实践,以及在算法落地过程中遇到的挑战和解决方案。 将从如下两个部分展开: AI算法在音乐推荐中的应用 音乐场景下的 AI 思考 从 2013 年 4 月正式上线至今,网易云音乐平台持续提供着:乐屏社区、UGC...
【技巧总结】位运算装逼指南
位算法的效率有多快我就不说,不信你可以去用 10 亿个数据模拟一下,今天给大家讲一讲位运算的一些经典例子。不过,最重要的不是看懂了这些例子就好,而是要在以后多去运用位运算这些技巧,当然,采用位运算,也是可以装逼的,不信,你往下看。我会从最简单的讲起,一道比一道难度递增,不过居然是讲技巧,那么也不会太难,相信你分分钟看懂。 判断奇偶数 判断一个数是基于还是偶数,相信很多人都做过,一般的做法的代码如下...
【管理系统课程设计】美少女手把手教你后台管理
【文章后台管理系统】URL设计与建模分析+项目源码+运行界面 栏目管理、文章列表、用户管理、角色管理、权限管理模块(文章最后附有源码) 1. 这是一个什么系统? 1.1 学习后台管理系统的原因 随着时代的变迁,现如今各大云服务平台横空出世,市面上有许多如学生信息系统、图书阅读系统、停车场管理系统等的管理系统,而本人家里就有人在用烟草销售系统,直接在网上完成挑选、购买与提交收货点,方便又快捷。 试想,若没有烟草销售系统,本人家人想要购买烟草,还要独自前往药...
4G EPS 第四代移动通信系统
目录 文章目录目录4G 与 LTE/EPCLTE/EPC 的架构E-UTRANE-UTRAN 协议栈eNodeBEPCMMES-GWP-GWHSSLTE/EPC 协议栈概览 4G 与 LTE/EPC 4G,即第四代移动通信系统,提供了 3G 不能满足的无线网络宽带化,主要提供数据(上网)业务。而 LTE(Long Term Evolution,长期演进技术)是电信领域用于手机及数据终端的高速无线通...
相关热词 c# clr dll c# 如何orm c# 固定大小的字符数组 c#框架设计 c# 删除数据库 c# 中文文字 图片转 c# 成员属性 接口 c#如何将程序封装 16进制负数转换 c# c#练手项目
立即提问

相似问题