keras model 训练 train_loss,train_acc再变,但是val_loss,val_test却一直不变,是哪里有问题?

Epoch 1/15
3112/3112 [==============================] - 73s 237ms/step - loss: 8.1257 - acc: 0.4900 - val_loss: 8.1763 - val_acc: 0.4927
Epoch 2/15
3112/3112 [==============================] - 71s 231ms/step - loss: 8.1730 - acc: 0.4929 - val_loss: 8.1763 - val_acc: 0.4927
Epoch 3/15
3112/3112 [==============================] - 72s 232ms/step - loss: 8.1730 - acc: 0.4929 - val_loss: 8.1763 - val_acc: 0.4427
Epoch 4/15
3112/3112 [==============================] - 71s 229ms/step - loss: 7.0495 - acc: 0.5617 - val_loss: 8.1763 - val_acc: 0.4927
Epoch 5/15
3112/3112 [==============================] - 71s 230ms/step - loss: 5.5504 - acc: 0.6549 - val_loss: 8.1763 - val_acc: 0.4927
Epoch 6/15
3112/3112 [==============================] - 71s 230ms/step - loss: 4.9359 - acc: 0.6931 - val_loss: 8.1763 - val_acc: 0.4927
Epoch 7/15
3112/3112 [==============================] - 71s 230ms/step - loss: 4.8969 - acc: 0.6957 - val_loss: 8.1763 - val_acc: 0.4927
Epoch 8/15
3112/3112 [==============================] - 72s 234ms/step - loss: 4.9446 - acc: 0.6925 - val_loss: 8.1763 - val_acc: 0.4927
Epoch 9/15
3112/3112 [==============================] - 71s 231ms/step - loss: 4.5114 - acc: 0.7201 - val_loss: 8.1763 - val_acc: 0.4927
Epoch 10/15
3112/3112 [==============================] - 73s 237ms/step - loss: 4.7944 - acc: 0.7021 - val_loss: 8.1763 - val_acc: 0.4927
Epoch 11/15
3112/3112 [==============================] - 74s 240ms/step - loss: 4.6789 - acc: 0.7095 - val_loss: 8.1763 - val_acc: 0.4927

1个回答

说明你的样本太少,明显已经过拟合了。

caozhy
贵阳老马马善福专业维修游泳池堵漏防水工程 回复weixin_42062762: 模型简单,某些权重趋向于0或者极大,再训练造成正向传播在特定上的样本结果不变,很正常
7 个月之前 回复
weixin_42062762
weixin_42062762 为什么过拟合验证分数一点不变呢!
7 个月之前 回复
Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
keras model.predict_classes() 问题
keras model.predict_classes()只能适用于sequential model, 对于Model 模型(functional model)该怎么达到类似的输出类别的效果 e.g results = list(model.predict_classes(data_test,verbose = 1)) score = accuracy_score(label_test,results) 这种在sequential model上可行的方法,如何在functional model 达到相同的效果
运行结果如下:train(generator,discriminator,gan_model,latent_dim) NameError: name 'train' is not defined,请问如何解决
import matplotlib.pyplot as plt from sklearn.model_selection import train_test_split from numpy import hstack from numpy import zeros from numpy import ones from numpy.random import rand from numpy.random import randn from keras.models import Sequential from keras.layers import Dense def define_discriminator(n_inputs=2): model=Sequential() model.add(Dense(25, activation='relu',kernel_initializer='he_uniform',input_dim=n_inputs)) model.add(Dense(1,activation='sigmoid')) model.compile(loss='binary_crossentropy',optimizer='adam', metrics=['accuracy']) return model def define_generator(latent_dim,n_outputs=2): model=Sequential() model.add(Dense(15, activation='relu',kernel_initializer='he_uniform', input_dim=latent_dim)) model.add(Dense(n_outputs,activation='linear')) return model def define_gan(generator,discriminator): discriminator.trainable=False model=Sequential() model.add(generator) model.add(discriminator) model.compile(loss='binary_crossentropy',optimizer='adam') return model def generate_real_samples(n): x1=rand(n)-0.5 x2=x1*x1 x1=x1.reshape(n,1) x2=x2.reshape(n,1) x=hstack((x1,x2)) y=ones((n,1)) return x,y def generate_latent_points(latent_dim,n): x_input=randn(latent_dim*n) x_input=x_input.reshape(n,latent_dim) return x_input def generate_fake_samples(generator,latent_dim,n): x_input=generate_latent_points(latent_dim,n) x=generator.predict(x_input) y=zeros((n,1)) return x,y def summarize_performance(epoch,generator,discriminator,latent_dim,n=100): x_real,y_real=generate_real_samples(n) _,acc_real=discriminator.evaluate(x_real,y_real,verbose=0) x_fake, y_fake = generate_fake_samples(generator,latent_dim,n) _, acc_fake = discriminator.evaluate(x_fake, y_fake, verbose=0) print(epoch,acc_real,acc_fake) plt.scatter(x_real[:,0],x_real[:,1],color='red') plt.scatter(x_fake[:, 0], x_fake[:, 1], color='blue') plt.show() def train(g_model,d_model,gan_model,latent_dim,n_epochs=10000,n_batch=128,n_eval=2000): half_batch=int(n_batch/2) for i in range(n_epochs): x_real,y_real=generate_real_samples(half_batch) x_fake,y_fake=generate_fake_samples(g_model,latent_dim,half_batch) d_model.train_on_batch(x_real,y_real) d_model.train_on_batch(x_fake, y_fake) x_gan=generate_latent_points(latent_dim,n_batch) y_gan=ones((n_batch,1)) gan_model.train_on_batch(x_gan,y_gan) if(i+1)%n_epochs==0: summarize_performance(i,g_model,d_model,latent_dim) latent_dim=5 discriminator=define_discriminator() generator=define_generator(latent_dim) gan_model=define_gan(generator,discriminator) train(generator,discriminator,gan_model,latent_dim) 问题
fashion_mnist识别准确率问题
fashion_mnist识别准确率一般为多少呢?我看好多人都是92%左右,但是我用一个网络达到了94%,想问问做过的小伙伴到底是多少? ``` #这是我的结果示意 x_shape: (60000, 28, 28) y_shape: (60000,) epoches: 0 val_acc: 0.4991 train_acc 0.50481665 epoches: 1 val_acc: 0.6765 train_acc 0.66735 epoches: 2 val_acc: 0.755 train_acc 0.7474 epoches: 3 val_acc: 0.7846 train_acc 0.77915 epoches: 4 val_acc: 0.798 train_acc 0.7936 epoches: 5 val_acc: 0.8082 train_acc 0.80365 epoches: 6 val_acc: 0.8146 train_acc 0.8107 epoches: 7 val_acc: 0.8872 train_acc 0.8872333 epoches: 8 val_acc: 0.896 train_acc 0.89348334 epoches: 9 val_acc: 0.9007 train_acc 0.8986 epoches: 10 val_acc: 0.9055 train_acc 0.90243334 epoches: 11 val_acc: 0.909 train_acc 0.9058833 epoches: 12 val_acc: 0.9112 train_acc 0.90868336 epoches: 13 val_acc: 0.9126 train_acc 0.91108334 epoches: 14 val_acc: 0.9151 train_acc 0.9139 epoches: 15 val_acc: 0.9172 train_acc 0.91595 epoches: 16 val_acc: 0.9191 train_acc 0.91798335 epoches: 17 val_acc: 0.9204 train_acc 0.91975 epoches: 18 val_acc: 0.9217 train_acc 0.9220333 epoches: 19 val_acc: 0.9252 train_acc 0.9234667 epoches: 20 val_acc: 0.9259 train_acc 0.92515 epoches: 21 val_acc: 0.9281 train_acc 0.9266667 epoches: 22 val_acc: 0.9289 train_acc 0.92826664 epoches: 23 val_acc: 0.9301 train_acc 0.93005 epoches: 24 val_acc: 0.9315 train_acc 0.93126667 epoches: 25 val_acc: 0.9322 train_acc 0.9328 epoches: 26 val_acc: 0.9331 train_acc 0.9339667 epoches: 27 val_acc: 0.9342 train_acc 0.93523335 epoches: 28 val_acc: 0.9353 train_acc 0.93665 epoches: 29 val_acc: 0.9365 train_acc 0.9379333 epoches: 30 val_acc: 0.9369 train_acc 0.93885 epoches: 31 val_acc: 0.9387 train_acc 0.9399 epoches: 32 val_acc: 0.9395 train_acc 0.9409 epoches: 33 val_acc: 0.94 train_acc 0.9417667 epoches: 34 val_acc: 0.9403 train_acc 0.94271666 epoches: 35 val_acc: 0.9409 train_acc 0.9435167 epoches: 36 val_acc: 0.9418 train_acc 0.94443333 epoches: 37 val_acc: 0.942 train_acc 0.94515 epoches: 38 val_acc: 0.9432 train_acc 0.9460667 epoches: 39 val_acc: 0.9443 train_acc 0.9468833 epoches: 40 val_acc: 0.9445 train_acc 0.94741666 epoches: 41 val_acc: 0.9462 train_acc 0.9482 epoches: 42 val_acc: 0.947 train_acc 0.94893336 epoches: 43 val_acc: 0.9472 train_acc 0.94946665 epoches: 44 val_acc: 0.948 train_acc 0.95028335 epoches: 45 val_acc: 0.9486 train_acc 0.95095 epoches: 46 val_acc: 0.9488 train_acc 0.9515833 epoches: 47 val_acc: 0.9492 train_acc 0.95213336 epoches: 48 val_acc: 0.9495 train_acc 0.9529833 epoches: 49 val_acc: 0.9498 train_acc 0.9537 val_acc: 0.9498 ``` ``` import tensorflow as tf from tensorflow import keras import numpy as np import matplotlib.pyplot as plt def to_onehot(y,num): lables = np.zeros([num,len(y)]) for i in range(len(y)): lables[y[i],i] = 1 return lables.T # 预处理数据 mnist = keras.datasets.fashion_mnist (train_images,train_lables),(test_images,test_lables) = mnist.load_data() print('x_shape:',train_images.shape) #(60000) print('y_shape:',train_lables.shape) X_train = train_images.reshape((-1,train_images.shape[1]*train_images.shape[1])) / 255.0 #X_train = tf.reshape(X_train,[-1,X_train.shape[1]*X_train.shape[2]]) Y_train = to_onehot(train_lables,10) X_test = test_images.reshape((-1,test_images.shape[1]*test_images.shape[1])) / 255.0 Y_test = to_onehot(test_lables,10) #双隐层的神经网络 input_nodes = 784 output_nodes = 10 layer1_nodes = 100 layer2_nodes = 50 batch_size = 100 learning_rate_base = 0.8 learning_rate_decay = 0.99 regularization_rate = 0.0000001 epochs = 50 mad = 0.99 learning_rate = 0.005 # def inference(input_tensor,avg_class,w1,b1,w2,b2): # if avg_class == None: # layer1 = tf.nn.relu(tf.matmul(input_tensor,w1)+b1) # return tf.nn.softmax(tf.matmul(layer1,w2) + b2) # else: # layer1 = tf.nn.relu(tf.matmul(input_tensor,avg_class.average(w1)) + avg_class.average(b1)) # return tf.matual(layer1,avg_class.average(w2)) + avg_class.average(b2) def train(mnist): X = tf.placeholder(tf.float32,[None,input_nodes],name = "input_x") Y = tf.placeholder(tf.float32,[None,output_nodes],name = "y_true") w1 = tf.Variable(tf.truncated_normal([input_nodes,layer1_nodes],stddev=0.1)) b1 = tf.Variable(tf.constant(0.1,shape=[layer1_nodes])) w2 = tf.Variable(tf.truncated_normal([layer1_nodes,layer2_nodes],stddev=0.1)) b2 = tf.Variable(tf.constant(0.1,shape=[layer2_nodes])) w3 = tf.Variable(tf.truncated_normal([layer2_nodes,output_nodes],stddev=0.1)) b3 = tf.Variable(tf.constant(0.1,shape=[output_nodes])) layer1 = tf.nn.relu(tf.matmul(X,w1)+b1) A2 = tf.nn.relu(tf.matmul(layer1,w2)+b2) A3 = tf.nn.relu(tf.matmul(A2,w3)+b3) y_hat = tf.nn.softmax(A3) # y_hat = inference(X,None,w1,b1,w2,b2) # global_step = tf.Variable(0,trainable=False) # variable_averages = tf.train.ExponentialMovingAverage(mad,global_step) # varible_average_op = variable_averages.apply(tf.trainable_variables()) #y = inference(x,variable_averages,w1,b1,w2,b2) cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=A3,labels=Y)) regularizer = tf.contrib.layers.l2_regularizer(regularization_rate) regularization = regularizer(w1) + regularizer(w2) +regularizer(w3) loss = cross_entropy + regularization * regularization_rate # learning_rate = tf.train.exponential_decay(learning_rate_base,global_step,epchos,learning_rate_decay) # train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss,global_step=global_step) train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss) # with tf.control_dependencies([train_step,varible_average_op]): # train_op = tf.no_op(name="train") correct_prediction = tf.equal(tf.argmax(y_hat,1),tf.argmax(Y,1)) accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) total_loss = [] val_acc = [] total_train_acc = [] x_Xsis = [] with tf.Session() as sess: tf.global_variables_initializer().run() for i in range(epochs): # x,y = next_batch(X_train,Y_train,batch_size) batchs = int(X_train.shape[0] / batch_size + 1) loss_e = 0. for j in range(batchs): batch_x = X_train[j*batch_size:min(X_train.shape[0],j*(batch_size+1)),:] batch_y = Y_train[j*batch_size:min(X_train.shape[0],j*(batch_size+1)),:] sess.run(train_step,feed_dict={X:batch_x,Y:batch_y}) loss_e += sess.run(loss,feed_dict={X:batch_x,Y:batch_y}) # train_step.run(feed_dict={X:x,Y:y}) validate_acc = sess.run(accuracy,feed_dict={X:X_test,Y:Y_test}) train_acc = sess.run(accuracy,feed_dict={X:X_train,Y:Y_train}) print("epoches: ",i,"val_acc: ",validate_acc,"train_acc",train_acc) total_loss.append(loss_e / batch_size) val_acc.append(validate_acc) total_train_acc.append(train_acc) x_Xsis.append(i) validate_acc = sess.run(accuracy,feed_dict={X:X_test,Y:Y_test}) print("val_acc: ",validate_acc) return (x_Xsis,total_loss,total_train_acc,val_acc) result = train((X_train,Y_train,X_test,Y_test)) def plot_acc(total_train_acc,val_acc,x): plt.figure() plt.plot(x,total_train_acc,'--',color = "red",label="train_acc") plt.plot(x,val_acc,color="green",label="val_acc") plt.xlabel("Epoches") plt.ylabel("acc") plt.legend() plt.show() ```
pycharm使用keras出现进度条信息多行打印
最近在用pycharm运行keras方面的代码时,会出现进度条多行打印问题,不知道是什么原因,但是我把代码放在Spyder上运行时,进度条是正常单行更新的,代码是深度学习的一个例程。在百度上也没搜到好的解决方法,恳请大家能帮忙解决这个问题, ``` from keras import layers,models from keras.datasets import mnist from keras.utils import to_categorical (train_images,train_labels),(test_images,test_labels) = mnist.load_data() train_images = train_images.reshape((60000,28,28,1)) train_images = train_images.astype('float32')/255 test_images = test_images.reshape((10000,28,28,1)) test_images = test_images.astype('float32')/255 train_labels = to_categorical(train_labels) test_labels = to_categorical(test_labels) model = models.Sequential() model.add(layers.Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1))) model.add(layers.MaxPool2D(2,2)) model.add(layers.Conv2D(64,(3,3),activation='relu')) model.add(layers.MaxPool2D(2,2)) model.add(layers.Conv2D(64,(3,3),activation='relu')) model.add(layers.Flatten()) model.add(layers.Dense(64,activation='relu')) model.add(layers.Dense(10,activation='softmax')) model.summary() model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy']) model.fit(train_images,train_labels,epochs=6,batch_size=64) #test_loss,test_acc = model.evaluate(test_images,test_labels) # print(test_loss,test_acc) ``` ![图片说明](https://img-ask.csdn.net/upload/201910/07/1570448232_727191.png)
tensorflow自定义的损失函数 focal_loss出现inf,在训练过程中出现inf
![图片说明](https://img-ask.csdn.net/upload/201905/05/1557048780_248292.png) ``` python def focal_loss(alpha=0.25, gamma=2.): """ focal loss used for train positive/negative samples rate out of balance, improve train performance """ def focal_loss_calc(y_true, y_pred): positive = tf.where(tf.equal(y_true, 1), y_pred, tf.ones_like(y_pred)) negative = tf.where(tf.equal(y_true, 0), y_pred, tf.zeros_like(y_pred)) return -(alpha*K.pow(1.-positive, gamma)*K.log(positive) + (1-alpha)*K.pow(negative, gamma)*K.log(1.-negative)) return focal_loss_calc ``` ```python self.keras_model.compile(optimizer=optimizer, loss=dice_focal_loss, metrics=[ mean_iou, dice_loss, focal_loss()]) ``` 上面的focal loss 开始还是挺正常的,随着训练过程逐渐减小大0.025左右,然后就突然变成inf。何解
在Spyder界面中使用tensorflow进行fashion_mnist数据集学习,结果loss为非数,并且准确率一直未变
1.建立了一个3个全连接层的神经网络; 2.代码如下: ``` import matplotlib as mpl import matplotlib.pyplot as plt #%matplotlib inline import numpy as np import sklearn import pandas as pd import os import sys import time import tensorflow as tf from tensorflow import keras print(tf.__version__) print(sys.version_info) for module in mpl, np, sklearn, tf, keras: print(module.__name__,module.__version__) fashion_mnist = keras.datasets.fashion_mnist (x_train_all, y_train_all), (x_test, y_test) = fashion_mnist.load_data() x_valid, x_train = x_train_all[:5000], x_train_all[5000:] y_valid, y_train = y_train_all[:5000], y_train_all[5000:] #tf.keras.models.Sequential model = keras.models.Sequential() model.add(keras.layers.Flatten(input_shape= [28,28])) model.add(keras.layers.Dense(300, activation="relu")) model.add(keras.layers.Dense(100, activation="relu")) model.add(keras.layers.Dense(10,activation="softmax")) ###sparse为最后输出为index类型,如果为one hot类型,则不需加sparse model.compile(loss = "sparse_categorical_crossentropy",optimizer = "sgd", metrics = ["accuracy"]) #model.layers #model.summary() history = model.fit(x_train, y_train, epochs=10, validation_data=(x_valid,y_valid)) ``` 3.输出结果: ``` runfile('F:/new/new world/deep learning/tensorflow/ex2/tf_keras_classification_model.py', wdir='F:/new/new world/deep learning/tensorflow/ex2') 2.0.0 sys.version_info(major=3, minor=7, micro=4, releaselevel='final', serial=0) matplotlib 3.1.1 numpy 1.16.5 sklearn 0.21.3 tensorflow 2.0.0 tensorflow_core.keras 2.2.4-tf Train on 55000 samples, validate on 5000 samples Epoch 1/10 WARNING:tensorflow:Entity <function Function._initialize_uninitialized_variables.<locals>.initialize_variables at 0x0000025EAB633798> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: WARNING: Entity <function Function._initialize_uninitialized_variables.<locals>.initialize_variables at 0x0000025EAB633798> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: 55000/55000 [==============================] - 3s 58us/sample - loss: nan - accuracy: 0.1008 - val_loss: nan - val_accuracy: 0.0914 Epoch 2/10 55000/55000 [==============================] - 3s 48us/sample - loss: nan - accuracy: 0.1008 - val_loss: nan - val_accuracy: 0.0914 Epoch 3/10 55000/55000 [==============================] - 3s 47us/sample - loss: nan - accuracy: 0.1008 - val_loss: nan - val_accuracy: 0.0914 Epoch 4/10 55000/55000 [==============================] - 3s 48us/sample - loss: nan - accuracy: 0.1008 - val_loss: nan - val_accuracy: 0.0914 Epoch 5/10 55000/55000 [==============================] - 3s 47us/sample - loss: nan - accuracy: 0.1008 - val_loss: nan - val_accuracy: 0.0914 Epoch 6/10 55000/55000 [==============================] - 3s 48us/sample - loss: nan - accuracy: 0.1008 - val_loss: nan - val_accuracy: 0.0914 Epoch 7/10 55000/55000 [==============================] - 3s 47us/sample - loss: nan - accuracy: 0.1008 - val_loss: nan - val_accuracy: 0.0914 Epoch 8/10 55000/55000 [==============================] - 3s 48us/sample - loss: nan - accuracy: 0.1008 - val_loss: nan - val_accuracy: 0.0914 Epoch 9/10 55000/55000 [==============================] - 3s 48us/sample - loss: nan - accuracy: 0.1008 - val_loss: nan - val_accuracy: 0.0914 Epoch 10/10 55000/55000 [==============================] - 3s 48us/sample - loss: nan - accuracy: 0.1008 - val_loss: nan - val_accuracy: 0.0914 ```
keras多GPU训练,其中一块无法调用
已经按multi_gpu_model进行了设置 但是运行的时候还是只能调用一个GPU,另一张计算卡完全没用,是什么原因呢? ``` from keras.utils import multi_gpu_model ... model = build_model() optimizer = keras.optimizers.Adadelta(lr=1.0, rho=0.95, epsilon=1e-06) model_parallel=multi_gpu_model(model,2) model_parallel.compile(loss='mse', optimizer=optimizer, metrics=['mae']) ... history = model_parallel.fit(train_data, y_train, epochs=EPOCHS, validation_split=0.2, verbose=1,callbacks=[PrintDot()]) ``` ![图片说明](https://img-ask.csdn.net/upload/201911/15/1573813427_19477.jpg) ![图片说明](https://img-ask.csdn.net/upload/201911/15/1573813436_730710.jpg)
基于keras,使用imagedatagenerator.flow函数读入数据,训练集ACC极低
在做字符识别的神经网络,数据集是用序号标好名称的图片,标签取图片的文件名。想用Imagedatagenrator 函数和flow函数,增加样本的泛化性,然后生成数据传入网络,可是这样acc=1/类别数,基本为零。请问哪里出了问题 ``` datagen = ImageDataGenerator( width_shift_range=0.1, height_shift_range=0.1 ) def read_train_image(self, name): myimg = Image.open(name).convert('RGB') return np.array(myimg) def train(self): #训练集 train_img_list = [] train_label_list = [] #测试集 test_img_list = [] test_label_list = [] for file in os.listdir('train'): files_img_in_array = self.read_train_image(name='train/' + file) train_img_list.append(files_img_in_array) # Image list add up train_label_list.append(int(file.split('_')[0])) # lable list addup for file in os.listdir('test'): files_img_in_array = self.read_train_image(name='test/' + file) test_img_list.append(files_img_in_array) # Image list add up test_label_list.append(int(file.split('_')[0])) # lable list addup train_img_list = np.array(train_img_list) train_label_list = np.array(train_label_list) test_img_list = np.array(train_img_list) test_label_list = np.array(train_label_list) train_label_list = np_utils.to_categorical(train_label_list, 5788) test_label_list = np_utils.to_categorical(test_label_list, 5788) train_img_list = train_img_list.astype('float32') test_img_list = test_img_list.astype('float32') test_img_list /= 255.0 train_img_list /= 255.0 ``` 这是图片数据的处理,图片和标签都存到list里。下面是用fit_genrator训练 ``` model.fit_generator( self.datagen.flow(x=train_img_list, y=train_label_list, batch_size=2), samples_per_epoch=len(train_img_list), epochs=10, validation_data=(test_img_list,test_label_list), ) ```
keras yolov3 tiny_yolo_body网络结构改为vgg16结构
将keras框架yolov3 tiny_yolo_body网络结构改为vgg16网络结构,程序能够运行 loss正常下降即可。
使用keras画出模型准确率评估的执行结果时出现:
建立好深度学习的模型后,使用反向传播法进行训练。 定义了训练方式: ``` model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=['accuracy']) ``` 执行训练: ``` train_history =model.fit(x=x_Train_normalize, y=y_Train_OneHot,validation_split=0.2, epochs=10,batch_size=200,verbose=2) ``` 执行后出现: ![图片说明](https://img-ask.csdn.net/upload/201910/17/1571243584_952792.png) 建立show_train_history显示训练过程: ``` import matplotlib.pyplot as plt def show_train_history(train_history,train,validation): plt.plot(train_history.history[train]) plt.plot(train_history.history[validation]) plt.title('Train History') plt.ylabel(train) plt.xlabel('Epoch') plt.legend(['train','validation'],loc='upper left') plt.show() ``` 画出准确率执行结果: ``` show_train_history(train_history,'acc','val_acc') ``` 结果出现以下问题: ![图片说明](https://img-ask.csdn.net/upload/201910/17/1571243832_179270.png) 这是怎么回事呀? 求求大佬救救孩子555
keras中model.evaluate()报错:'numpy.float64' object is not iterable
``` x_train, x_test, y_train, y_test = train_test_split(x_data, y_data, test_size=0.25) mean = x_train.mean(axis=0) std = x_train.std(axis=0) train_data = (x_train - mean) / std test_data = (x_test - mean) / std model = Sequential([Dense(64, input_shape=(6,)), Activation('relu'), Dense(32), Activation('relu'), Dense(1)]) sgd = keras.optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True) model.compile(loss='mean_squared_error', optimizer=sgd) k = model.fit [loss, sgd] = model.evaluate(test_data, y_test, verbose=1) ``` 最后一步不知道哪出了问题。。test_data, y_test都是dataframe啊 TypeError Traceback (most recent call last) <ipython-input-29-3b0767a3c446> in <module> ----> 1 [loss, mse] = model.evaluate(test_data, y_test, verbose=1) TypeError: 'numpy.float64' object is not iterable
keras 训练 IMDB数据 为什么预测的是正面情感?
学习 利用Keras中的IMDB数据集,对评论进行二分类,有个疑问是:为什么预测的是正面情感?代码如下: from keras.datasets import imdb from keras import models from keras import layers import numpy as np import matplotlib.pyplot as plt def vectorize_sequences(sequences, dimension=10000): results = np.zeros((len(sequences), dimension)) for i, sequence in enumerate(sequences): results[i, sequence] = 1. print('i=',i,'results[i]=',results[i]) return results (train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000) '''word_index = imdb.get_word_index() reverse_word_index = dict([(value, key) for (key, value) in word_index.items()]) decoded_review = ' '.join([reverse_word_index.get(i - 3, '?') for i in train_data[0]]) ''' x_train = vectorize_sequences(train_data) x_test = vectorize_sequences(test_data) y_train = np.asarray(train_labels).astype('float32') y_test = np.asarray(test_labels).astype('float32') model = models.Sequential() model.add(layers.Dense(16, activation='relu', input_shape=(10000,))) model.add(layers.Dense(16,activation='relu')) model.add(layers.Dense(1,activation='sigmoid')) x_val = x_train[:10000] partial_x_train = x_train[10000:] y_val = y_train[:10000] partial_y_train = y_train[10000:] model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc']) history = model.fit(partial_x_train, partial_y_train, epochs=20, batch_size=512,validation_data=(x_val, y_val)) history_dict = history.history loss_value = history_dict['loss'] val_loss_value = history_dict['val_loss'] epochs = range(1,len(loss_value)+1) plt.plot(epochs, loss_value, 'bo', label='Trianing Loss') plt.plot(epochs, val_loss_value, 'b', label='Validation Loss') plt.title('Training and validation loss') plt.xlabel('Epochs') plt.ylabel('Loss') plt.legend() plt.show()
使用keras搭建黑体汉字单个字符识别网络val_acc=0.0002
这是我读入训练数据的过程,数据集是根据txt文件生成的单个汉字的图像(shape为64*64)788种字符(包括数字和X字符),每张图片的开头命名为在txt字典中的位置,作为标签, ``` def read_train_image(self, name): img = Image.open(name).convert('RGB') return np.array(img) def train(self): train_img_list = [] train_label_list = [] for file in os.listdir('train'): files_img_in_array = self.read_train_image(name='train/'+ file) train_img_list.append(files_img_in_array) # Image list add up train_label_list.append(int(file.split('_')[0])) # lable list addup train_img_list = np.array(train_img_list) train_label_list = np.array(train_label_list) train_label_list = np_utils.to_categorical(train_label_list, self.count) train_img_list = train_img_list.astype('float32') train_img_list /= 255 ``` 训练下来,虽然train_acc达到0.99,但是验证accuracy一直都等于0. 下面是网络结构: ``` model = Sequential() #创建第一个卷积层。 model.add(Convolution2D(32, 3, 3, border_mode='valid', input_shape=(64,64,3),kernel_regularizer=l2(0.0001))) model.add(BatchNormalization(axis=3)) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) #创建第二个卷积层。 model.add(Convolution2D(64, 3, 3, border_mode='valid',kernel_regularizer=l2(0.0001))) model.add(BatchNormalization()) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) #创建第三个卷积层。 model.add(Convolution2D(128, 3, 3, border_mode='valid',kernel_regularizer=l2(0.0001))) model.add(BatchNormalization()) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) # 创建全连接层。 model.add(Flatten()) model.add(Dense(128, init= 'he_normal')) model.add(BatchNormalization()) model.add(Activation('relu')) #创建输出层,使用 Softmax函数输出属于各个字符的概率值。 model.add(Dense(output_dim=self.count, init= 'he_normal')) model.add(Activation('softmax')) #设置神经网络中的损失函数和优化算法。 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=['accuracy']) #开始训练,并设置批尺寸和训练的步数。 model.fit( train_img_list, train_label_list, epochs=500, batch_size=128, validation_split=0.2, verbose=1, shuffle= False, ) ``` 大概结构是这样,十几轮后训练集acc达到了0.99.验证集val_acc为0.网上说这种情况大概是过拟合了,希望高手指点一下。
用keras做图像2分类,结果总是所有test样本归为其中一类?
用keras做图像2分类,label非平衡,约1:10,代码如下: data = np.load('D:/a.npz') image_data, label_data= data['image'], data['label'] 由于数据不平衡,用分层K折拆分为3组, train_x=image_data[train] test_x=image_data[test] train_y=label_data[train] test_y=label_data[test] train_x = np.array(train_x) test_x = np.array(test_x) train_x = train_x.reshape(train_x.shape[0],1,28,28) test_x = test_x.reshape(test_x.shape[0],1,28,28) train_x = train_x.astype('float32') test_x = test_x.astype('float32') train_x /=255 test_x /=255 train_y = np.array(train_y) test_y = np.array(test_y) 然后用keras的序贯模型 model.compile(optimizer='rmsprop',loss="binary_crossentropy",metrics=['acc']) model.fit(train_x, train_y,batch_size=128, class_weight = 'auto', epochs=10,verbose=1,validation_data=(test_x, test_y)) from sklearn.metrics import confusion_matrix y_pred_model = model.predict_proba(test_x) C=confusion_matrix(test_y,y_pred_model) print(C) 结果总是所有test样本归为一类, 推测可能是不平衡,模型认为最优化就是将所有样本都认作为较大类,但是将2分类label改为1:1后,结果仍然是所有test都归为一类: [[22 0] [21 0]] 请教这是啥原因?代码错在哪?
keras 训练网络时出现ValueError
rt 使用keras中的model.fit函数进行训练时出现错误:ValueError: None values not supported. 错误信息如下: ``` File "C:/Users/Desktop/MNISTpractice/mnist.py", line 93, in <module> model.fit(x_train,y_train, epochs=2, callbacks=callback_list,validation_data=(x_val,y_val)) File "C:\Anaconda3\lib\site-packages\keras\engine\training.py", line 1575, in fit self._make_train_function() File "C:\Anaconda3\lib\site-packages\keras\engine\training.py", line 960, in _make_train_function loss=self.total_loss) File "C:\Anaconda3\lib\site-packages\keras\legacy\interfaces.py", line 87, in wrapper return func(*args, **kwargs) File "C:\Anaconda3\lib\site-packages\keras\optimizers.py", line 432, in get_updates m_t = (self.beta_1 * m) + (1. - self.beta_1) * g File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\math_ops.py", line 820, in binary_op_wrapper y = ops.convert_to_tensor(y, dtype=x.dtype.base_dtype, name="y") File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 639, in convert_to_tensor as_ref=False) File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 704, in internal_convert_to_tensor ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref) File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\constant_op.py", line 113, in _constant_tensor_conversion_function return constant(v, dtype=dtype, name=name) File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\constant_op.py", line 102, in constant tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape)) File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 360, in make_tensor_proto raise ValueError("None values not supported.") ValueError: None values not supported. ```
keras结果ACC: 1.0000 Recall: 1.0000 F1-score: 1.0000 Precesion: 1.0000的原因?
用keras做的图像2分类,仅仅跑了5个epoch, 结果: [[205 0] [ 0 28]] keras的AUC为: 1.0 AUC: 1.0000 ACC: 1.0000 Recall: 1.0000 F1-score: 1.0000 Precesion: 1.0000 代码: data = np.load('.npz') image_data, label_data= data['image'], data['label'] skf = StratifiedKFold(n_splits=3, shuffle=True) for train, test in skf.split(image_data, label_data): train_x=image_data[train] test_x=image_data[test] train_y=label_data[train] test_y=label_data[test] train_x = np.array(train_x) test_x = np.array(test_x) train_x = train_x.reshape(train_x.shape[0],1,28,28) test_x = test_x.reshape(test_x.shape[0],1,28,28) train_x = train_x.astype('float32') test_x = test_x.astype('float32') train_x /=255 test_x /=255 train_y = np.array(train_y) test_y = np.array(test_y) model.compile(optimizer='rmsprop',loss="binary_crossentropy",metrics=["accuracy"]) model.fit(train_x, train_y,batch_size=64,epochs=5,verbose=1,validation_data=(test_x, test_y)]) 从结果看,代码存在离谱的错误,请教各位专家,错在哪?谢谢
Tensorflow 2.0 : When using data tensors as input to a model, you should specify the `steps_per_epoch` argument.
下面代码每次执行到epochs 中的最后一个step 都会报错,请教大牛这是什么问题呢? ``` import tensorflow_datasets as tfds dataset, info = tfds.load('imdb_reviews/subwords8k', with_info=True, as_supervised=True) train_dataset,test_dataset = dataset['train'],dataset['test'] tokenizer = info.features['text'].encoder print('vocabulary size: ', tokenizer.vocab_size) sample_string = 'Hello world, tensorflow' tokenized_string = tokenizer.encode(sample_string) print('tokened id: ', tokenized_string) src_string= tokenizer.decode(tokenized_string) print(src_string) for t in tokenized_string: print(str(t) + ': '+ tokenizer.decode([t])) BUFFER_SIZE=6400 BATCH_SIZE=64 num_train_examples = info.splits['train'].num_examples num_test_examples=info.splits['test'].num_examples print("Number of training examples: {}".format(num_train_examples)) print("Number of test examples: {}".format(num_test_examples)) train_dataset=train_dataset.shuffle(BUFFER_SIZE) train_dataset=train_dataset.padded_batch(BATCH_SIZE,train_dataset.output_shapes) test_dataset=test_dataset.padded_batch(BATCH_SIZE,test_dataset.output_shapes) def get_model(): model=tf.keras.Sequential([ tf.keras.layers.Embedding(tokenizer.vocab_size,64), tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)), tf.keras.layers.Dense(64,activation='relu'), tf.keras.layers.Dense(1,activation='sigmoid') ]) return model model =get_model() model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) import math #from tensorflow import keras #train_dataset= keras.preprocessing.sequence.pad_sequences(train_dataset, maxlen=BUFFER_SIZE) history =model.fit(train_dataset, epochs=2, steps_per_epoch=(math.ceil(BUFFER_SIZE/BATCH_SIZE) -90 ), validation_data= test_dataset) ``` Train on 10 steps Epoch 1/2 9/10 [==========================>...] - ETA: 3s - loss: 0.6955 - accuracy: 0.4479 --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-111-8ddec076c096> in <module> 6 epochs=2, 7 steps_per_epoch=(math.ceil(BUFFER_SIZE/BATCH_SIZE) -90 ), ----> 8 validation_data= test_dataset) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs) 726 max_queue_size=max_queue_size, 727 workers=workers, --> 728 use_multiprocessing=use_multiprocessing) 729 730 def evaluate(self, /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_arrays.py in fit(self, model, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, **kwargs) 672 validation_steps=validation_steps, 673 validation_freq=validation_freq, --> 674 steps_name='steps_per_epoch') 675 676 def evaluate(self, /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_arrays.py in model_iteration(model, inputs, targets, sample_weights, batch_size, epochs, verbose, callbacks, val_inputs, val_targets, val_sample_weights, shuffle, initial_epoch, steps_per_epoch, validation_steps, validation_freq, mode, validation_in_fit, prepared_feed_values_from_dataset, steps_name, **kwargs) 437 validation_in_fit=True, 438 prepared_feed_values_from_dataset=(val_iterator is not None), --> 439 steps_name='validation_steps') 440 if not isinstance(val_results, list): 441 val_results = [val_results] /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_arrays.py in model_iteration(model, inputs, targets, sample_weights, batch_size, epochs, verbose, callbacks, val_inputs, val_targets, val_sample_weights, shuffle, initial_epoch, steps_per_epoch, validation_steps, validation_freq, mode, validation_in_fit, prepared_feed_values_from_dataset, steps_name, **kwargs) 174 if not is_dataset: 175 num_samples_or_steps = _get_num_samples_or_steps(ins, batch_size, --> 176 steps_per_epoch) 177 else: 178 num_samples_or_steps = steps_per_epoch /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_arrays.py in _get_num_samples_or_steps(ins, batch_size, steps_per_epoch) 491 return steps_per_epoch 492 return training_utils.check_num_samples(ins, batch_size, steps_per_epoch, --> 493 'steps_per_epoch') 494 495 /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_utils.py in check_num_samples(ins, batch_size, steps, steps_name) 422 raise ValueError('If ' + steps_name + 423 ' is set, the `batch_size` must be None.') --> 424 if check_steps_argument(ins, steps, steps_name): 425 return None 426 /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_utils.py in check_steps_argument(input_data, steps, steps_name) 1199 raise ValueError('When using {input_type} as input to a model, you should' 1200 ' specify the `{steps_name}` argument.'.format( -> 1201 input_type=input_type_str, steps_name=steps_name)) 1202 return True 1203 ValueError: When using data tensors as input to a model, you should specify the `steps_per_epoch` argument.
使用keras进行分类问题时,验证集loss,accuracy 显示0.0000e+00,但是最后画图像时能显示出验证曲线
data_train, data_test, label_train, label_test = train_test_split(data_all, label_all, test_size= 0.2, random_state = 1) data_train, data_val, label_train, label_val = train_test_split(data_train,label_train, test_size = 0.25) data_train = np.asarray(data_train, np.float32) data_test = np.asarray(data_test, np.float32) data_val = np.asarray(data_val, np.float32) label_train = np.asarray(label_train, np.int32) label_test = np.asarray(label_test, np.int32) label_val = np.asarray(label_val, np.int32) training = model.fit_generator(datagen.flow(data_train, label_train_binary, batch_size=200,shuffle=True), validation_data=(data_val,label_val_binary), samples_per_epoch=len(data_train)*8, nb_epoch=30, verbose=1) def plot_history(history): plt.plot(training.history['acc']) plt.plot(training.history['val_acc']) plt.title('model accuracy') plt.xlabel('epoch') plt.ylabel('accuracy') plt.legend(['acc', 'val_acc'], loc='lower right') plt.show() plt.plot(training.history['loss']) plt.plot(training.history['val_loss']) plt.title('model loss') plt.xlabel('epoch') plt.ylabel('loss') plt.legend(['loss', 'val_loss'], loc='lower right') plt.show() plot_history(training) ![图片说明](https://img-ask.csdn.net/upload/201812/10/1544423669_112599.jpg)![图片说明](https://img-ask.csdn.net/upload/201812/10/1544423681_598605.jpg)
如果将keras情感分析模型应用到Java Web上,那Web后台怎么预处理字符串并转化为特征向量?
尚属初学折腾了一点简单的代码,但是很想知道怎么将训练模型应用到Web项目上。 训练模型时用了如下代码: ```python # 加载数据内容步骤省略 from keras.preprocessing.text import Tokenizer from keras.preprocessing.sequence import pad_sequences tokenizer = Tokenizer() tokenizer.fit_on_texts(train_texts) train_sequences = tokenizer.texts_to_sequences(train_texts) test_sequences = tokenizer.texts_to_sequences(test_texts) train_data = pad_sequences(train_sequences, maxlen=MAX_SEQUENCE_LENGTH) test_data = pad_sequences(test_sequences, maxlen=MAX_SEQUENCE_LENGTH) ``` 因为tokenizer使用训练文本进行fit,记录了词典之类的信息,那我要在Web项目上调用模型预测文本的之前是否应该再用之前tokenizer里的信息去做预处理才对?那需要如此处理的话我在Web后台该怎么做?
Java学习的正确打开方式
在博主认为,对于入门级学习java的最佳学习方法莫过于视频+博客+书籍+总结,前三者博主将淋漓尽致地挥毫于这篇博客文章中,至于总结在于个人,实际上越到后面你会发现学习的最好方式就是阅读参考官方文档其次就是国内的书籍,博客次之,这又是一个层次了,这里暂时不提后面再谈。博主将为各位入门java保驾护航,各位只管冲鸭!!!上天是公平的,只要不辜负时间,时间自然不会辜负你。 何谓学习?博主所理解的学习,它是一个过程,是一个不断累积、不断沉淀、不断总结、善于传达自己的个人见解以及乐于分享的过程。
程序员必须掌握的核心算法有哪些?
由于我之前一直强调数据结构以及算法学习的重要性,所以就有一些读者经常问我,数据结构与算法应该要学习到哪个程度呢?,说实话,这个问题我不知道要怎么回答你,主要取决于你想学习到哪些程度,不过针对这个问题,我稍微总结一下我学过的算法知识点,以及我觉得值得学习的算法。这些算法与数据结构的学习大多数是零散的,并没有一本把他们全部覆盖的书籍。下面是我觉得值得学习的一些算法以及数据结构,当然,我也会整理一些看过...
有哪些让程序员受益终生的建议
从业五年多,辗转两个大厂,出过书,创过业,从技术小白成长为基层管理,联合几个业内大牛回答下这个问题,希望能帮到大家,记得帮我点赞哦。 敲黑板!!!读了这篇文章,你将知道如何才能进大厂,如何实现财务自由,如何在工作中游刃有余,这篇文章很长,但绝对是精品,记得帮我点赞哦!!!! 一腔肺腑之言,能看进去多少,就看你自己了!!! 目录: 在校生篇: 为什么要尽量进大厂? 如何选择语言及方...
大学四年自学走来,这些私藏的实用工具/学习网站我贡献出来了
大学四年,看课本是不可能一直看课本的了,对于学习,特别是自学,善于搜索网上的一些资源来辅助,还是非常有必要的,下面我就把这几年私藏的各种资源,网站贡献出来给你们。主要有:电子书搜索、实用工具、在线视频学习网站、非视频学习网站、软件下载、面试/求职必备网站。 注意:文中提到的所有资源,文末我都给你整理好了,你们只管拿去,如果觉得不错,转发、分享就是最大的支持了。 一、电子书搜索 对于大部分程序员...
linux系列之常用运维命令整理笔录
本博客记录工作中需要的linux运维命令,大学时候开始接触linux,会一些基本操作,可是都没有整理起来,加上是做开发,不做运维,有些命令忘记了,所以现在整理成博客,当然vi,文件操作等就不介绍了,慢慢积累一些其它拓展的命令,博客不定时更新 free -m 其中:m表示兆,也可以用g,注意都要小写 Men:表示物理内存统计 total:表示物理内存总数(total=used+free) use...
比特币原理详解
一、什么是比特币 比特币是一种电子货币,是一种基于密码学的货币,在2008年11月1日由中本聪发表比特币白皮书,文中提出了一种去中心化的电子记账系统,我们平时的电子现金是银行来记账,因为银行的背后是国家信用。去中心化电子记账系统是参与者共同记账。比特币可以防止主权危机、信用风险。其好处不多做赘述,这一层面介绍的文章很多,本文主要从更深层的技术原理角度进行介绍。 二、问题引入 假设现有4个人...
程序员接私活怎样防止做完了不给钱?
首先跟大家说明一点,我们做 IT 类的外包开发,是非标品开发,所以很有可能在开发过程中会有这样那样的需求修改,而这种需求修改很容易造成扯皮,进而影响到费用支付,甚至出现做完了项目收不到钱的情况。 那么,怎么保证自己的薪酬安全呢? 我们在开工前,一定要做好一些证据方面的准备(也就是“讨薪”的理论依据),这其中最重要的就是需求文档和验收标准。一定要让需求方提供这两个文档资料作为开发的基础。之后开发...
网页实现一个简单的音乐播放器(大佬别看。(⊙﹏⊙))
今天闲着无事,就想写点东西。然后听了下歌,就打算写个播放器。 于是乎用h5 audio的加上js简单的播放器完工了。 演示地点演示 html代码如下` music 这个年纪 七月的风 音乐 ` 然后就是css`*{ margin: 0; padding: 0; text-decoration: none; list-...
Python十大装B语法
Python 是一种代表简单思想的语言,其语法相对简单,很容易上手。不过,如果就此小视 Python 语法的精妙和深邃,那就大错特错了。本文精心筛选了最能展现 Python 语法之精妙的十个知识点,并附上详细的实例代码。如能在实战中融会贯通、灵活使用,必将使代码更为精炼、高效,同时也会极大提升代码B格,使之看上去更老练,读起来更优雅。
数据库优化 - SQL优化
以实际SQL入手,带你一步一步走上SQL优化之路!
2019年11月中国大陆编程语言排行榜
2019年11月2日,我统计了某招聘网站,获得有效程序员招聘数据9万条。针对招聘信息,提取编程语言关键字,并统计如下: 编程语言比例 rank pl_ percentage 1 java 33.62% 2 cpp 16.42% 3 c_sharp 12.82% 4 javascript 12.31% 5 python 7.93% 6 go 7.25% 7 p...
通俗易懂地给女朋友讲:线程池的内部原理
餐盘在灯光的照耀下格外晶莹洁白,女朋友拿起红酒杯轻轻地抿了一小口,对我说:“经常听你说线程池,到底线程池到底是个什么原理?”
《奇巧淫技》系列-python!!每天早上八点自动发送天气预报邮件到QQ邮箱
将代码部署服务器,每日早上定时获取到天气数据,并发送到邮箱。 也可以说是一个小型人工智障。 知识可以运用在不同地方,不一定非是天气预报。
经典算法(5)杨辉三角
杨辉三角 是经典算法,这篇博客对它的算法思想进行了讲解,并有完整的代码实现。
英特尔不为人知的 B 面
从 PC 时代至今,众人只知在 CPU、GPU、XPU、制程、工艺等战场中,英特尔在与同行硬件芯片制造商们的竞争中杀出重围,且在不断的成长进化中,成为全球知名的半导体公司。殊不知,在「刚硬」的背后,英特尔「柔性」的软件早已经做到了全方位的支持与支撑,并持续发挥独特的生态价值,推动产业合作共赢。 而对于这一不知人知的 B 面,很多人将其称之为英特尔隐形的翅膀,虽低调,但是影响力却不容小觑。 那么,在...
腾讯算法面试题:64匹马8个跑道需要多少轮才能选出最快的四匹?
昨天,有网友私信我,说去阿里面试,彻底的被打击到了。问了为什么网上大量使用ThreadLocal的源码都会加上private static?他被难住了,因为他从来都没有考虑过这个问题。无独有偶,今天笔者又发现有网友吐槽了一道腾讯的面试题,我们一起来看看。 腾讯算法面试题:64匹马8个跑道需要多少轮才能选出最快的四匹? 在互联网职场论坛,一名程序员发帖求助到。二面腾讯,其中一个算法题:64匹...
面试官:你连RESTful都不知道我怎么敢要你?
干货,2019 RESTful最贱实践
为啥国人偏爱Mybatis,而老外喜欢Hibernate/JPA呢?
关于SQL和ORM的争论,永远都不会终止,我也一直在思考这个问题。昨天又跟群里的小伙伴进行了一番讨论,感触还是有一些,于是就有了今天这篇文。 声明:本文不会下关于Mybatis和JPA两个持久层框架哪个更好这样的结论。只是摆事实,讲道理,所以,请各位看官勿喷。 一、事件起因 关于Mybatis和JPA孰优孰劣的问题,争论已经很多年了。一直也没有结论,毕竟每个人的喜好和习惯是大不相同的。我也看...
白话阿里巴巴Java开发手册高级篇
不久前,阿里巴巴发布了《阿里巴巴Java开发手册》,总结了阿里巴巴内部实际项目开发过程中开发人员应该遵守的研发流程规范,这些流程规范在一定程度上能够保证最终的项目交付质量,通过在时间中总结模式,并推广给广大开发人员,来避免研发人员在实践中容易犯的错误,确保最终在大规模协作的项目中达成既定目标。 无独有偶,笔者去年在公司里负责升级和制定研发流程、设计模板、设计标准、代码标准等规范,并在实际工作中进行...
SQL-小白最佳入门sql查询一
不要偷偷的查询我的个人资料,即使你再喜欢我,也不要这样,真的不好;
redis分布式锁,面试官请随便问,我都会
文章有点长并且绕,先来个图片缓冲下! 前言 现在的业务场景越来越复杂,使用的架构也就越来越复杂,分布式、高并发已经是业务要求的常态。像腾讯系的不少服务,还有CDN优化、异地多备份等处理。 说到分布式,就必然涉及到分布式锁的概念,如何保证不同机器不同线程的分布式锁同步呢? 实现要点 互斥性,同一时刻,智能有一个客户端持有锁。 防止死锁发生,如果持有锁的客户端崩溃没有主动释放锁,也要保证锁可以正常释...
项目中的if else太多了,该怎么重构?
介绍 最近跟着公司的大佬开发了一款IM系统,类似QQ和微信哈,就是聊天软件。我们有一部分业务逻辑是这样的 if (msgType = "文本") { // dosomething } else if(msgType = "图片") { // doshomething } else if(msgType = "视频") { // doshomething } else { // doshom...
Nginx 原理和架构
Nginx 是一个免费的,开源的,高性能的 HTTP 服务器和反向代理,以及 IMAP / POP3 代理服务器。Nginx 以其高性能,稳定性,丰富的功能,简单的配置和低资源消耗而闻名。 Nginx 的整体架构 Nginx 里有一个 master 进程和多个 worker 进程。master 进程并不处理网络请求,主要负责调度工作进程:加载配置、启动工作进程及非停升级。worker 进程负责处...
Python 编程开发 实用经验和技巧
Python是一门很灵活的语言,也有很多实用的方法,有时候实现一个功能可以用多种方法实现,我这里总结了一些常用的方法和技巧,包括小数保留指定位小数、判断变量的数据类型、类方法@classmethod、制表符中文对齐、遍历字典、datetime.timedelta的使用等,会持续更新......
YouTube排名第一的励志英文演讲《Dream(梦想)》
Idon’t know what that dream is that you have, I don't care how disappointing it might have been as you've been working toward that dream,but that dream that you’re holding in your mind, that it’s po...
“狗屁不通文章生成器”登顶GitHub热榜,分分钟写出万字形式主义大作
一、垃圾文字生成器介绍 最近在浏览GitHub的时候,发现了这样一个骨骼清奇的雷人项目,而且热度还特别高。 项目中文名:狗屁不通文章生成器 项目英文名:BullshitGenerator 根据作者的介绍,他是偶尔需要一些中文文字用于GUI开发时测试文本渲染,因此开发了这个废话生成器。但由于生成的废话实在是太过富于哲理,所以最近已经被小伙伴们给玩坏了。 他的文风可能是这样的: 你发现,...
程序员:我终于知道post和get的区别
是一个老生常谈的话题,然而随着不断的学习,对于以前的认识有很多误区,所以还是需要不断地总结的,学而时习之,不亦说乎
《程序人生》系列-这个程序员只用了20行代码就拿了冠军
你知道的越多,你不知道的越多 点赞再看,养成习惯GitHub上已经开源https://github.com/JavaFamily,有一线大厂面试点脑图,欢迎Star和完善 前言 这一期不算《吊打面试官》系列的,所有没前言我直接开始。 絮叨 本来应该是没有这期的,看过我上期的小伙伴应该是知道的嘛,双十一比较忙嘛,要值班又要去帮忙拍摄年会的视频素材,还得搞个程序员一天的Vlog,还要写BU...
加快推动区块链技术和产业创新发展,2019可信区块链峰会在京召开
11月8日,由中国信息通信研究院、中国通信标准化协会、中国互联网协会、可信区块链推进计划联合主办,科技行者协办的2019可信区块链峰会将在北京悠唐皇冠假日酒店开幕。   区块链技术被认为是继蒸汽机、电力、互联网之后,下一代颠覆性的核心技术。如果说蒸汽机释放了人类的生产力,电力解决了人类基本的生活需求,互联网彻底改变了信息传递的方式,区块链作为构造信任的技术有重要的价值。   1...
Java世界最常用的工具类库
Apache Commons Apache Commons有很多子项目 Google Guava 参考博客
相关热词 c# 图片上传 c# gdi 占用内存 c#中遍历字典 c#控制台模拟dos c# 斜率 最小二乘法 c#进程延迟 c# mysql完整项目 c# grid 总行数 c# web浏览器插件 c# xml 生成xsd
立即提问