{"title":"An Evolutionary Statistical Learning Theory","authors":"Sung-Hae Jun, Kyung-Whan Oh","volume":12,"journal":"International Journal of Computer and Information Engineering","pagesStart":3873,"pagesEnd":3881,"ISSN":"1307-6892","URL":"https:\/\/publications.waset.org\/pdf\/5739","abstract":"Statistical learning theory was developed by Vapnik. It\r\nis a learning theory based on Vapnik-Chervonenkis dimension. It also\r\nhas been used in learning models as good analytical tools. In general, a\r\nlearning theory has had several problems. Some of them are local\r\noptima and over-fitting problems. As well, statistical learning theory\r\nhas same problems because the kernel type, kernel parameters, and\r\nregularization constant C are determined subjectively by the art of\r\nresearchers. So, we propose an evolutionary statistical learning theory\r\nto settle the problems of original statistical learning theory.\r\nCombining evolutionary computing into statistical learning theory,\r\nour theory is constructed. We verify improved performances of an\r\nevolutionary statistical learning theory using data sets from KDD cup.","references":"[1] A. Ben-Hur, A. D. Horn, H. Siegelmann, V. Vapnik, \"Support Vector\r\nClustering,\" Journal of Machine Learning Research 2, 2001, pp.\r\n125-137.\r\n[2] L. Breiman, J. H. Friedman, R. A. Olshen, C. J. Stone, Classification and\r\nRegression Trees, Wadsworth Inc., 1984.\r\n[3] J. Cannady, \"Artificial Neural Networks for Misuse Detection. National\r\nInformation Systems,\" Proceedings of Security Conference, 1998.\r\n[4] G. Casella, R. L. Berger, Statistical Inference, Duxbury Press, 1990.\r\n[5] V. Cherkassky, F. Mulier, Learning From Data Concepts, Theory, and\r\nMethods, John Wiley & Sons, 1998.\r\n[6] R. Cooley, B. Mobasher, J. Srivastava, \"Web Mining: Information and\r\nPattern Discovery on the World Wide Web,\" Proceeding of the 9th IEEE\r\nInternational Conference on Tools with Artificial Intelligence, 1997.\r\n[7] R. Cooley, P. N. Tan, J. Srivastava, \"Discovery of interesting usage\r\npatterns from web data,\" Technical Report TR 99-022, University of\r\nMinnesota, 1999.\r\n[8] H. Debar, M. Becke, D. Siboni, \"A Neural Network Component for an\r\nIntrusion Detection System,\" Proceedings of the IEEE Computer Society\r\nSymposium on Research in Security and Privacy, 1992, pp. 240-250.\r\n[9] H. Debar, B. Dorizzi, \"An Application of a Recurrent Network to an\r\nIntrusion Detection System,\" Proceedings of the International Joint\r\nConference on Neural Networks, 1992, pp 78-483.\r\n[10] A. E. Eiben, J. E. Smith, Introduction to Evolutionary Computing,\r\nSpringer, 2003.\r\n[11] S. M. Emran, M. Xu, N. Ye, Q. Chen, X. Li, \"Probabilistic techniques for\r\nintrusion detection based on computer audit data,\" IEEE Transactions on\r\nSystems, Man and Cybernetics, Part A, vol.31, 2001, pp.266-274.\r\n[12] D. Fisher, K. Hildrum, J. Hong, M. Newman, M. Thomas, R. Vuduc,\r\n\"SWAMI: A Frame-work for Collaborative Filtering Algorithm\r\nDevelopment and Evaluation,\" Proceeding of SIGIR 2000, ACM Press,\r\n2000.\r\n[13] D. B. Fogel, Evolutionary Computation, IEEE Press, 1995.\r\n[14] L. J. Fogel, A. J. Owens, M. J. Walsh, Artificial Intelligence through\r\nSimulated Evolution, Wiley, Chichester, UK, 1996.\r\n[15] A. K. Ghosh, Learning Program Behavior Profiles for Intrusion\r\nDetection, USENIX, 1999.\r\n[16] J. W. Haines, R. P. Lippmann, D. J. Fried, M. A. Zissman, E. Tran, S. B.\r\nBoswell, \"1999 DARPA Intrusion Detection Evaluation: Design and\r\nProcedures,\" Technical Report 1062, Lincoln Laboratory, MIT, 2001.\r\n[17] S. Haykin, Neural Networks, Prentice Hall, 1999.\r\n[18] S. Huet, A. Bouvier, M. A. Poursat, E. Jolivet, Statistical Tools for\r\nNonlinear Regression, Springer Series in Statistics, Springer, 2003.\r\n[19] S. H. Jun, \"Hybrid Statistical Learning Model for Intrusion Detection of\r\nNetworks,\" The KIPS Transaction: Part C, vol. 10-C, no. 6, 2003, pp.\r\n705-710.\r\n[20] S. H. Jun, \"Web Usage Mining Using Support Vector Machine,\" Lecture\r\nNote in Computer Science, vol. 3512, 2005, pp. 349-356.\r\n[21] S. Kumar, E. H. Spafford, \"An Application of Pattern Matching in\r\nIntrusion Detection,\" Technical Report CSD-TR-94-013, Purdue\r\nUniversity, 1994.\r\n[22] W. Lee, S. J. Stolfo, K. W. Mok, \"A data mining framework for building\r\nintrusion detection models,\" Proceedings of the 1999 IEEE Symposium\r\non Security and Privacy, 1999, pp.120-132.\r\n[23] R. J. A. Little, D. B. Rubin, Statistical Analysis with Missing Data, Wiley\r\nInter-Science, 2002.\r\n[24] B. Liu, \"Fuzzy Random Chance-Constrained Programming,\" IEEE\r\nTransactions on Fuzzy Systems, vol. 9, Issue 5, 2001, pp. 713-720.\r\n[25] J. Luo, S. M. Bridges, \"Mining Fuzzy Association Rules and Fuzzy\r\nFrequency Episodes for Intrusion Detection,\" International Journal of\r\nIntelligent Systems, John Wiley & Sons, 2000, pp. 687-703.\r\n[26] G. Mclachlan, D. Peel, Finite Mixture Models, John Wiley & Sons, Inc.,\r\n2000.\r\n[27] T. M. Mitchell, Machine Learning, McGraw-Hill, 1997.\r\n[28] T. M. Mitchell, An introduction to Genetic Algorithms, MIT Press, 1998.\r\n[29] S. Mukkamala, G. Janoski, A. Sung, \"Intrusion Detection Using Neural\r\nNetworks and Support Vector Machines,\" Proceedings of International\r\nSymposium on Applications and the Internet Technology, 2000, pp.\r\n209-216\r\n[30] R. H. Myers, Classical and Modern Regression with Applications,\r\nDuxbury Press, 1990.\r\n[31] A. T. Quang, Q. L. Zhang, X. Li, \"Evolving Support Vector Machine\r\nParameters,\" Proceedings of the First International Conference on\r\nMachine Learning and Cybernetics, 2002, pp. 548-551.\r\n[32] J. Ryan, M. J. Lin, R. Miikkulainen, \"Intrusion Detection with Neural\r\nNetworks,\" Advances in Neural Information Processing Systems 10,\r\nCambridge, MA: MIT Press, 1998.\r\n[33] A. J. Smola, Regression estimation with support vector learning\r\nmachines, Master-s thesis, Technische University, 1996.\r\n[34] V. Vapnik, Statistical Learning Theory, John Wiley & Sons, Inc., 1998.\r\n[35] X. Yao, \"Evolving Artificial Neural Networks,\" Proceedings of the IEEE,\r\nvol. 87, Issue 9, 1999, pp. 1423-1447.\r\n[36] http:\/\/www.ecn.purdue.edu\/KDDCUP\r\n[37] http:\/\/www.ll.mit.edu\/IST\/ideval\/data","publisher":"World Academy of Science, Engineering and Technology","index":"Open Science Index 12, 2007"}