分布式solr-cloud之solr部署
准备:solr-4.10.2.tgztomcat三台虚拟机:192.168.184.128 solr1192.168.184.129 solr2192.168.184.130 solr31:解压tomcat解压到/usr/local/solr-tomcat2:解压 tar -zxvf solr-4.10.2.tgz并且把 solr-4.10.2/exam
准备:
solr-4.10.2.tgz
tomcat
三台虚拟机:
192.168.184.128 solr1
192.168.184.129 solr2
192.168.184.130 solr3
1:解压tomcat解压到/usr/local/solr-tomcat
2:解压 tar -zxvf solr-4.10.2.tgz
并且把 solr-4.10.2/example/webapps/solr.war 解压至 /usr/local/webapps/solr 目录下
并且把solr-4.10.2\example\lib\ext 下的 jar 包放到 solr\WEB-INF\lib 下 (这里主要是一些log4j的架包,还可以在solr\WEB-INF下创建一个class文件里面放log4配置文件这样就可以配置日志打印输出了)
以192.168.184.128 solr1为例创建文件夹
mkdir -p /usr/local/solrCloud/config-files
mkdir -p /usr/local/solrCloud/solr-lib
把 solr/WEB-INF/lib 下的所有 jar 包拷贝到/usr/local/solrCloud/solr-lib目录
cp -av /usr/local/webapps/solr/WEB-INF/lib*.jar /usr/local/solrCloud/solr-lib
solr/example/solr/collection1/conf 下的所有文件拷贝到 /usr/local/solrCloud/config-files 目录
cp -av /usr/local/webapps/solr/example/solr/collection1/conf/* /usr/local/solrCloud/config-files
将 /usr/local/webapps/solr目录拷贝到 /usr/local/solr-tomcat/webapps/目录下面
cp -av /usr/local/webapps/solr /usr/local/solr-tomcat/webapps
</pre><pre name="code" class="plain"></pre></p><p>将 <span style="color: rgb(51, 51, 51); font-family: 'Open Sans', 'Helvetica Neue', Helvetica, Arial, STHeiti, 'Microsoft Yahei', sans-serif; font-size: 14px; line-height: 22.3999996185303px;">/usr/local/webapps/solr</span>目录拷贝到 /usr/local/solr-tomcat/webapps/目录下面<span style="font-family: 'Open Sans', 'Helvetica Neue', Helvetica, Arial, STHeiti, 'Microsoft Yahei', sans-serif; color: rgb(51, 51, 51);"><span style="font-size: 14px; line-height: 22.3999996185303px;"><span style="line-height: 22.3999996185303px;"></span></span></span></p><p></p><pre name="code" class="plain">cp -av /usr/local/webapps/solr /usr/local/solr-tomcat/webapps
创建/usr/local/solr-cores目录在该目录下创建solr.xml文件,该文件非常重要,以后是用来配置solrcore的
vim solr.xml
内容如下
<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
<logging enabled="true">
<watcher size="100" threshold="INFO" />
</logging>
<cores defaultCoreName="collection1" adminPath="/admin/cores" host="${host:}" hostPort="8080" hostContext="${hostContext:solr}" zkClientTimeout="${zkClientTimeout:15000}">
</cores>
</solr>
创建/usr/local/solr-tomcat/conf/Catalina 目录 和 /usr/local/solr-tomcat/conf/Catalina/localhost 目录
在/usr/local/solr-tomcat/conf/Catalina/localhost目录下创建solr.xml, 此文件为 Solr/home 的配置文件
内容如下
<?xml version="1.0" encoding="UTF-8"?>
<Context docBase="/usr/local/solr-tomcat/webapps/solr" debug="0" crossContext="true">
<Environment name="solr/home" type="java.lang.String" value="/usr/local/solr-cores" override="true"/>
</Context>
修改 tomcat/bin/cataina.sh ,加入以下内容
JAVA_OPTS="$JAVA_OPTS -DzkHost=solr1:2181,solr2:2181,solr3:2181"
以上配置同步到另外两台服务器
scp -r /usr/local/solr-bin/ root@solr2:/usr/local/
scp -r /usr/local/solr-conf/ root@solr2:/usr/local/
scp -r /usr/local/solrCloud/ root@solr2:/usr/local/
scp -r /usr/local/solr-cores/ root@solr2:/usr/local/
scp -r /usr/local/solr-tomcat/ root@sol2:/usr/local/
scp -r /usr/local/solr-bin/ root@solr3:/usr/local/
scp -r /usr/local/solr-conf/ root@solr3:/usr/local/
scp -r /usr/local/solrCloud/ root@solr3:/usr/local/
scp -r /usr/local/solr-cores/ root@solr3:/usr/local/
scp -r /usr/local/solr-tomcat/ root@solr3:/usr/local/
SolrCloud 是通过 ZooKeeper 集群来保证配置文件的变更及时同步到各个节点上,所以,需要将配置文件上传到 ZooKeeper 集群中:执行如下操作(以下ip均可使用域名进行操作)
java -classpath .:/usr/local/solrCloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost solr1:2181,solr2:2181,solr3:2181 -confdir /usr/local/solrCloud/config-files/ -confname myconf
校验 zookeeper 的配置文件
[root@hadoop1 bin]# /usr/local/zookeeper/bin/zkCli.sh -server solr1:2181
Connecting to solr1:2181
2015-07-01 15:04:44,509 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-07-01 15:04:44,518 [myid:] - INFO [main:Environment@100] - Client environment:host.name=<NA>
2015-07-01 15:04:44,518 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_31
2015-07-01 15:04:44,522 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2015-07-01 15:04:44,522 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/local/jdk/jdk1.8.0_31/jre
2015-07-01 15:04:44,525 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../build/lib/*.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper/bin/../zookeeper-3.4.6.jar:/usr/local/zookeeper/bin/../src/java/lib/*.jar:/usr/local/zookeeper/bin/../conf:.:/usr/local/jdk/jdk1.8.0_31/lib/dt.jar:/usr/local/jdk/jdk1.8.0_31/lib/tools.jar
2015-07-01 15:04:44,526 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2015-07-01 15:04:44,526 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2015-07-01 15:04:44,526 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2015-07-01 15:04:44,527 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2015-07-01 15:04:44,527 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2015-07-01 15:04:44,527 [myid:] - INFO [main:Environment@100] - Client environment:os.version=2.6.32-504.el6.x86_64
2015-07-01 15:04:44,528 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2015-07-01 15:04:44,528 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2015-07-01 15:04:44,528 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/usr/local/solr-tomcat/bin
2015-07-01 15:04:44,532 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=solr1:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
Welcome to ZooKeeper!
2015-07-01 15:04:44,607 [myid:] - INFO [main-SendThread(solr1:2181):ClientCnxn$SendThread@975] - Opening socket connection to server solr1/192.168.184.128:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
[zk: solr1:2181(CONNECTING) 0] 2015-07-01 15:04:44,793 [myid:] - INFO [main-SendThread(solr1:2181):ClientCnxn$SendThread@852] - Socket connection established to solr1/192.168.184.128:2181, initiating session
2015-07-01 15:04:44,812 [myid:] - INFO [main-SendThread(solr1:2181):ClientCnxn$SendThread@1235] - Session establishment complete on server solr1/192.168.184.128:2181, sessionid = 0x14e47ba8aa30002, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
ls /
[configs, zookeeper, overseer, aliases.json, live_nodes, collections, overseer_elect, clusterstate.json]
[zk: solr1:2181(CONNECTED) 1] ls /
[configs, zookeeper, overseer, aliases.json, live_nodes, collections, overseer_elect, clusterstate.json]
这里只要有configs, zookeeper文件即可
启动tomcat
分别在三台机子上运行
/usr/local/solr-tomcat/bin/startup.sh
创建 Collection 及初始 Shard
curl 'http://192.168.184.128:8080/solr/admin/collections?action=CREATE&name=collection2&instanceDir=collection2&numShards=3&replicationFactor=1'
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrServer;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.impl.CloudSolrServer;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrDocument;
import org.apache.solr.common.SolrDocumentList;
import org.apache.solr.common.SolrInputDocument;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
public class SolrCloudTest {
public static final Log LOG = LogFactory.getLog(SolrCloudTest.class);
private static CloudSolrServer cloudSolrServer;
private static synchronized CloudSolrServer getCloudSolrServer(final String zkHost) {
LOG.info("connect to :"+zkHost+"\n");
if(cloudSolrServer == null) {
try {
cloudSolrServer = new CloudSolrServer(zkHost);
}catch(Exception e) {
e.printStackTrace();
}
}
return cloudSolrServer;
}
private void addIndex(SolrServer solrServer) {
try {
Collection<SolrInputDocument> docs = new ArrayList<SolrInputDocument>();
for (int i = 0;i<=200;i++){
SolrInputDocument doc = new SolrInputDocument();
String key = "";
key = String.valueOf(i);
doc.addField("id", key);
doc.addField("test_s", key+"value");
docs.add(doc);
}
LOG.info("docs info:"+docs+"\n");
solrServer.add(docs);
solrServer.commit();
}catch(SolrServerException e) {
System.out.println("Add docs Exception !!!");
e.printStackTrace();
}catch(IOException e){
e.printStackTrace();
}catch (Exception e) {
System.out.println("Unknowned Exception!!!!!");
e.printStackTrace();
}
}
public void search(SolrServer solrServer, String Str) {
SolrQuery query = new SolrQuery();
query.setRows(20);
query.setQuery(Str);
try {
LOG.info("query string: "+ Str);
System.out.println("query string: "+ Str);
QueryResponse response = solrServer.query(query);
SolrDocumentList docs = response.getResults();
System.out.println(docs);
System.out.println(docs.size());
System.out.println("doc num:" + docs.getNumFound());
System.out.println("elapse time:" + response.getQTime());
for (SolrDocument doc : docs) {
String area = (String) doc.getFieldValue("test_s");
String id = (String) doc.getFieldValue("id");
System.out.println("id: " + id);
System.out.println("tt_s: " + area);
System.out.println();
}
} catch (SolrServerException e) {
e.printStackTrace();
} catch(Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
final String zkHost = "192.168.184.128:2181";
final String defaultCollection = "collection2";
final int zkClientTimeout = 10000;
final int zkConnectTimeout = 10000;
CloudSolrServer cloudSolrServer = getCloudSolrServer(zkHost);
cloudSolrServer.setDefaultCollection(defaultCollection);
cloudSolrServer.setZkClientTimeout(zkClientTimeout);
cloudSolrServer.setZkConnectTimeout(zkConnectTimeout);
try{
cloudSolrServer.connect();
System.out.println("connect solr cloud zk sucess");
} catch (Exception e){
LOG.error("connect to collection "+defaultCollection+" error\n");
System.out.println("error message is:"+e);
e.printStackTrace();
System.exit(1);
}
SolrCloudTest solrt = new SolrCloudTest();
try{
solrt.addIndex(cloudSolrServer);
} catch(Exception e){
e.printStackTrace();
}
solrt.search(cloudSolrServer, "id:*"); ;
cloudSolrServer.shutdown();
}
}
成功插入数据
该文章参考http://segmentfault.com/a/1190000000595712了
更多推荐
所有评论(0)